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PREFACE 


On  the  10th  and  11th  of  August  1982,  the  Naval  Training 
Equipment  Center  was  the  host  organization  for  a  Workshop  on 
Instructional  Features  and  Instructor/Operator  Station  Design 
for  Training  Systems  sponsored  by  the  Chief  of  Naval  Research* 
The  goal  of  the  workshop  was  to  review  current  research  and 
development  work  related  to  the  operation  of  the  Instructor  s 
station  of  training  systems  particularly  with  an  eye  for 
developing  functional  specifications  for  these  stations. 
Historically  much  of  the  research  and  development  work  has 
been  sponsored  by  the  Naval  Air  Systems  Command  and  the 
Air  Force,  but  we  feel  that  many  of  the  developments  are 
applicable  to  a  variety  of  simulation-based  training  situ¬ 
ations. 

Several  people  deserve  thanks  for  their  help  in  making 
the  workshop  possible.  CDR  P.  M.  Curran  of  the  Office  of 
Naval  Technology  cheerfully  encouraged  us  and  provided 
support,  and  locally  at  the  Human  Factors  Laboratory  of  the 
Naval  Training  Equipment  Center,  Catherine  Bottelman  and 
Hilda  Worsham  both  performed  the  many  chores  necessary 
during  the  workshop  and  later  edited  the  papers  and  pre¬ 
pared  the  report  for  printing.  We  thank  them  all. 
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COMMENTS  ON  INSTRUCTIONAL  FEATURES 


GILBERT  L.  RICARD 
Naval  Training  Equipment  Center 


Ouring  the  past  decade  we  have  seen  dra¬ 
matic  changes  of  the  role  instructors  play  in 
device-based  training  and  of  the  training 
tasks  to  which  simulation  has  been  applied. 

The  main  change  we  have  seen  has  been  the  ex¬ 
tension  of  synthetic  training  to  tasks  which 
simulation  previously  could  not  support.  Much 
of  this  has  been  due  to  the  development  of 
software  and  hardware  to  create  visual  dis¬ 
plays,  the  application  of  new  means  of  inter¬ 
acting  with  computer-based  trainers,  and  the 
proliferation  of  small  computer  systems  with 
large  amounts  of  memory.  Of  course,  some  of 
this  change  has  been  the  result  of  an  in¬ 
creased  reliance  on  simulation  for  training— 
both  by  military  training  commands  and  ci¬ 
vilian  organizations— but  most  has  resulted 
from  the  greatly  increased  capabilities  of  the 
devices  themselves.  All  change  seems  to  have 
its  good  and  bad  aspects,  and  while  the  ex¬ 
tension  of  simulator-based  training  to  complex 
tasks  is  most  welcome,  our  notions  of  how  in¬ 
structional  personnel  should  interact  with 
these  new  trainers  have  not  kept  pace  with 
hardware  development.  Three  trends  have  con¬ 
tributed  to  this  problem: 

1.  Device-based  flight  training  has 
developed  from  the  simulations  of  takeoff  and 
landing  of  a  decade  ago  to  almost  all  aspects 
of  the  tactical  use  of  aircraft.  Largely, 
this  result  has  come  from  the  development  of 
simulations  of  sensors— radar,  sonar,  forward- 
looking  infrared,  low-light  level  TV,  and 
computer-generated  visual  scenes— which  allow 
all  members  of  a  crew  to  interact  in  the  same 
training  scenario.  Thus,  individual  proce¬ 
dural  training  has  expanded  to  tactical  team 
training  where  more  than  one  trainee's  actions 
must  be  monitored  by  instructional  personnel. 

As  the  number  of  trainees  participating  In  a 
training  event  increases,  so  also  does  the 
number  of  instructors  needed.  The  resulting 
requirement  for  the  design  of  training  equip¬ 
ment  has  been  to  reduce  the  workload  and  sup¬ 
port  the  communication  of  these  Instructors. 
Partly  this  Is  to  make  their  jobs  easier  (both 
to  do  and  to  learn),  and  partly  this  Is  to 
enable  them  to  act  more  effectively  as  In¬ 
structors. 

2.  The  instructor/operator  station  Is 
the  Interface  Instructional  personnel  have  to 
a  training  device  and  the  services  It  pro¬ 
vides.  The  past  decade  has  seen  a  good  deal 
of  change  In  these  consoles.  The  repeater 
Instruments  and  knobs  and  dials  once  common 
have  been  replaced  by  cathode  ray  tubes,  alpha¬ 
numeric  and  multifunction  keyboards.  Joy¬ 
sticks,  and  the  like.  Today  training  tasks 
are  selected  by  entering  data  In  response  to 


choices  presented  in  menu  form  on  programmable 
displays  and  light  pens  are  used  to  select 
many  of  the  options.  On  the  horizon,  we  see 
flat  panels  with  touch  sensitive  surfaces  re¬ 
placing  much  of  the  input/output  hardware-  in 
current  use.  Such  systems  present  in  succes¬ 
sive  displays  the  information  and  choices  that 
previously  were  spread  over  console  surfaces. 
These  systems  pose  the  design  problem  of  re¬ 
casting  the  rules  of  the  spatial  grouping  of 
information  and  choices  into  successive  tem¬ 
poral  groupings.  In  the  future  we  expect  that 
a  single  surface  will  serve  to  present  infor¬ 
mation  and  record  inputs,  and  system  designers 
shall  have  to  be  both  selective  and  efficient 
in  their  choice  of  what  to  display  at  a  given 
time  and  what  information  to  require  from  in¬ 
structors.  Rules,  or  at  least  guidelines, 
will  have  to  be  developed  for  moving  about  in 
menu-driven  displays,  for  recovering  from 
incorrect  inputs,  and  for  keeping  track  of 
choices  that  have  been  made.  This  problem  of 
engineering  the  flow  of  events  which  present 
information  to  instructors  and  allow  them  to 
exert  control  over  a  device  seems  to  be  at  the 
heart  of  deficiencies  in  today's  complex 
trainers.  In  future  systems,  probably  no¬ 
where  will  the  conflicting  requirements  of 
allowing  for  flexible  usage  and  of  providing 
for  standardized  instruction  need  to  be  more 
delicately  balanced  than  at  the  operator's 
Interface  to  the  device. 

3.  The  past  decade  has  seen  great  in¬ 
creases  of  the  processing  power  and  storage 
capability  of  computer  systems— all  at  ever 
falling  prices.  It  has  become  common  to  see 
training  devices  with  several  minicomputers 
networked  to  process  an  appropriately  parsed 
problem,  and  we  expect  that  the  future  will 
only  see  more  of  this  as  microcomputers  are 
devoted  to  smaller  and  smaller  elements  of  the 
Information  processing  and  display  tasks. 

This  is  true  now  for  many  of  the  calculations 
Involved  in  a  simulation  and  will  be  true  also 
for  the  services  a  device  offers  its  users. 
From  an  instructional  point  of  view,  these 
developments  allow  possibilities  previously 
considered  Impracticable— where  the  computer 
system  of  a  trainer  not  only  supports  the 
simulation  of  a  training  task  but  also  stores 
and  manipulates  data  useful  for  instructional 
control.  Today's  computers  are  not  only  data 
processors  but  machines  to  store  and  operate 
upon  data  bases  as  well  as  engines  that  can 
manipulate  symbolic  relations  and  make  infer¬ 
ences.  These  new  capabilities,  emerging  as 
software  with  supporting  hardware,  will  allow 
training  systems  undreamed  of  a  decade  ago. 

The  problem  for  design  then  Is  to  define  how 
such  services  should  operate  and  how  to  fit 
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them  Into  operational  training  devices. 


y:- 


These  three  trends  have  resulted  In  the 
requirement  to  define  better  the  instructional 
role  a  training  device's  computer  capability 
should  fulfill  and  to  define  functionally  the 
services  a  training  device  should  support. 
Research  and  development  efforts  have  stressed 
tne  observation  that  devices  should  provide 
not  only  a  simulation  of  a  task  but  should 
also  be  equipment  to  support  the  training  of 
that  task.  Much  of  the  research  work  has  ex¬ 
amined  instructional  features  that  have  been 
incorporated  into  current  devices  or  has  sug¬ 
gested  new  features.  Presently  instructional 
features  fall  into  two  classes:  those  de¬ 
signed  to  support  real-time  instruction  using 
a  device  and  those  designed  to  provide  off¬ 
line  services  for  instructional  or  administra¬ 
tive  personnel.  Many  of  the  real-time  train¬ 
ing  features  are  old  friends  such  as  the  abil¬ 
ity  to  freeze  a  task,  set  environmental  con¬ 
ditions,  move  vehicles  and  the  like.  Re¬ 
search  efforts  have  produced  others  such  as 
models  to  determine  the  actions  of  targets  or 
the  output  of  voice  synthesizers,  computer 
measurement  of  a  trainee's  progress,  and 
schemes  to  automate  feedback  for  training. 
Features  designed  for  off-line  support  tend  to 
be  relatively  new  because  the  storage  required 
for  them  has  only  recently  become  available. 
Here  we  have  seen  developments  in  the  keeping 
of  students'  records,  in  providing  briefing 
and  debriefing  information,  in  the  incorpora¬ 
tion  of  software  to  tutor  instructors  on  the 
use  of  a  device,  and  in  programs  to  define  and 
create  new  training  exercises.  While  neither 
of  these  lists  is  inclusive,  they  do  provide  a 
flavor  of  the  developments  we  expect  to  see  in 
the  training  devices  of  the  future. 


S  Every  so  often  it  is  prudent  to  review  the 

ways  we  do  things  and  the  technology  we  employ 
to  do  them.  From  problems  uncovered  in  train¬ 
ers  currently  operational,  it  is  clear  that 
now  is  the  time  to  examine  the  functional  de¬ 
sign  of  the  instructor/operator  stations  of 
training  equipment.  We  need  to  be  able  to 
£  take  advantage  of  new  technology  by  knowing 

"J  how  to  design  it  well  so  that  it  can  support 

t-.  training  easily.  The  goal  of  this  workshop 

k  then  is  to  provide  information  for  the  devel- 

opment  of  new  systems  as  well  as  to  present  an 
overview  of  the  knowledge  currently  available. 


Large  and  growing  literature  exists  on 
>i|  many  of  the  topics  related  to  the  design  of 

^  instructor/operator  stations.  These  include 

/  topics  such  as  instructional  systems  devel- 

opment,  workplace  design,  and  the  utilization 
of  training  devices  as  well  as  discussions  of 
r'  the  design  of  man-computer  dialogues  and  de- 

scriptions  of  display  and  control  equipment  for 
^  man-machine  interfaces.  Only  a  small  portion 

■,  of  this  writing  is  devoted  to  the  design  and 

operation  of  training  equipment,  and  for  the 
most  part  it  exists  as  a  technical  literature 


which  may  not  be  widely  distributed.  Because 
of  this,  a  bibliography  follows  that  lists 
many  of  the  reports  discussing  instructional 
features  and  console  operation  particularly 
for  training  applications. 
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INTRODUCTION 

The  Operations  Training  Division  of 
the  Air  Force  Hunan  Resources  Laboratory 
(AFHRL/OT)  has  as  Its  ulsslon  the  laprove- 
aent  of  Air  Force  caabat  effectiveness 
through  training  related  science  and  tech¬ 
nology.  Located  at  Hllllaas  Air  Force 
Base,  Arizona,  AFHRL/OT  performs  this 
Mission  through  two  primary  functions: 
behavioral  RIO  to  solve  flying  training 
problem  through  Improved  technology;  and, 
engineering  RIO  to  develop  training  devices 
as  vehicles  for  training  RID.  The  person¬ 
nel  who  support  these  two  areas  fora  a 
diverse,  Multidisciplinary  teaa  of 
specialists  ranging  froa  research  psycholo¬ 
gists.  research  Instructor  pilots,  and 
hiaaan  factors  specialists  to  aerospace 
engineers,  aatheaatl clans,  and  coaputer 
prograaaers.  AFHRL/OT  operates  and  Main¬ 
tains  two  of  the  nation's  most  advanced 
simulation  facilities  for  training  and  RID, 
the  Advanced  Simulator  for  Pilot  Training 
(ASPT1  at  Williams  Air  Force  Base,  and  the 
Simulator  for  Alr-to-AIr  Caabat  (SAAC)  at 
Luke  Air  Force  Base.  At  present,  the  ASPT 
Is  configured  with  an  F-16  and  an  A-10 
cockpit,  and  the  SAAC  provides  two  F-4 
cockpits  for  simulated  air-to-air  coabat. 

AFHRL/OT  RID  efforts  have  been  accca- 
pllshed  In  a  variety  of  areas  Including 
simulator  training  effectiveness,  training 
methods,  pl’ot  performance  measureaent,  and 
pilot  perception  and  cognition.  In  the 
area  of  simulator  training  effectiveness, 
efforts  have  been  directed  toward  the 
development  and  assessment  of  visual 
display  systems  and  display  generation 
techniques  Including  helmet-mounted  visual 
displays  and  computer  generated  Imagery; 
the  evaluation  of  alternative  motion  cueing 
systems  such  as  the  G-seat,  G-suit,  and 
cockpit  motion  platform;  and,  the  deter¬ 
mination  of  visual  display  requirements 
Including  f lei d-of -view,  co’or,  resolution, 
aerial  perspective,  and  terrain  cues.  As 
for  the  area  of  training  methods,  efforts 
have  addressed,  for  example,  whole  vs. 
part-task  training  using  full -mission 
flight  slmu’ators  and  desk-top.  special- 
function  trainers.  The  area  of  pl’ot 
performance  measurement  has  also  been 
strongly  supported  and  Includes  experiments 
on  measures  of  basic  flying  and  combat 
skills,  pilot  workload,  and  stress.  In  the 
area  of  pilot  perception  and  cognition,  a 
variety  of  pilot  factors  have  been  examined 


such  as  decision  making,  tactical  planning, 
and  visual  cue  utilization. 

Because  the  efficacy  of  aircrew  training 
Is  dependent  not  only  on  the  training  capa¬ 
bilities  of  flight  slaulators  but  also  on 
the  quality  of  flight  Instruction,  AFHRL/OT 
has  applied  considerable  resources  to  the 
design  and  evaluation  of  Instructor/ 
operator  station  (IOS)  controls,  displays, 
and  workstation  layouts  as  well  as  the 
evaluation  of  various  10S  configurations. 
The  specific  efforts  that  were  accomplished 
by  AFHRL/OT,  and  the  activities  In 
progress,  are  as  follows. 

AFHRL/OT  IOS  RIO  ACCOMPLISHMENTS 

1.  A-10  Flight  Simulator  IPS.  An  IOS 
was  designed  for  the  Air  Force  A-10  Opera¬ 
tional  Flight  Trainer.  This  effort  Involved 
a  determination  of  the  IOS  controls  and 
displays  required  for  Instructing  simulated 
A-10  flight  and  the  design  of  an  A-10  IOS 
consistent  with  these  requirements.  In 
this  RID  effort  »  group  of  A-lO  instructor 
pilots  (IPs!  was  observed  during  the  con¬ 
duct  of  flight  simulator  training  In  the 
Advanced  Simulator  for  Pilot  Training  (ASPT) 
and  each  IP  was  administered  a  questionnaire 
when  the  training  was  concluded.  The 
questionnaire  was  designed  to  assess  what 
controls  and  displays  were  used  to  Instruct 
the  student  pilots  and  what  should  be  added 
to  the  IOS  to  facilitate  flight  Inst.uctlon. 
Both  the  observational  and  questionnaire 
data  were  analyzed  to  determine  the  control 
and  display  requirements  for  an  A-10  simu¬ 
lator  IOS.  An  IOS  was  subsequently  designed 
which  Incorporated  these  controls  and  dis¬ 
plays.  In  general,  the  IOS  consists  of 
seven  Cathode  Ray  Tubes  (CRTs),  A-10 
repeater  Instruments,  a  variety  of  controls 
and  indicators,  and  a  CRT  hardcopy  unit. 
Two  of  the  CRTs  are  video  monitors  which 
would  be  used  to  display  the  student 
pilot's  out-the-wl ndow  visual  scene,  and 
one  CRT  Is  a  closed  circuit  television 
monitor  that  would  enable  the  IP  to  monitor 
the  In-cockpit  activities  of  the  student 
pilot.  The  remaining  CRTs  would  provide 
Informational  feedback  about  the  student's 
flight  performance.  A  variety  of  displays 
were  developed  for  these  CRTs  and  the 
function  and  operation  of  each  display  were 
determined.  The  IOS  also  included  A-10 
repeater  instruments  which  were  arranged 
the  same  as  they  are  In  the  actual  aircraft. 
Additionally,  a  number  of  controls  were 
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incorporated  in  the  IOS  for  the  executive 
and  administrative  control  of  the  training 
process.  The  specific  configuration  of 
these  controls  was  provided  and  the 
function,  operation,  and  layout  of  the 
controls  were  specified.  This  IOS  design 
effort  was  fully  described  by  Gray,  Chun, 
Warner,  and  Eubanks  C981). 

?.  A-10  Training  Features  Evaluation. 
The  utility  of  various  instructional  fea- 
tures  of  the  ASPT  IOS  were  evaluated  In  the 
context  of  A-10  flight  training.  Six  of 
the  features  available  on  the  ASPT  IOS  were 
evaluated.  They  were:  problem  and  para¬ 
meter  freeze,  rapid  initialization,  auto¬ 
matic  demonstration,  automated  performance 
feedback,  self-confrontation,  and  task 
difficulty.  In  the  study,  IPs  were  required 
to  rate  the  utility  of  each  instructional 
feature  and  a  systematic  observational 
procedure  was  used  to  determine  the  fre - 
quency  with  which  each  feature  was  used  by 
the  IPs.  The  data  analysis  indicated  that 
performance  feedback  and  Initialization 
were  rated  highest  and  were  used  most 
frequently.  Problem  and  parameter  freeze 
was  a1  so  used  regularly.  Many  of  the 
features,  such  as  automated  performance 
feedback,  task  difficulty,  preprogramed 
demonstrations,  and  self -confrontation, 
were  rarely  used.  Some  of  the  features, 
such  as  freeze  and  Initialization,  were  not 
used  In  an  instructional  manner,  but  more 
for  training  management  purposes.  That  is, 
they  tended  to  be  used  to  terminate  a 
training  exercise  and  to  provide,  a  transi¬ 
tion  to  the  next  exercise,  rather  than  for 
providing  opportunities  to  give  the  student 
pilots  performance  feedback  and  the 
appropriate  remediation.  This  study  Is 
documented  In  the  report  by  Gray,  Chun, 
Warner,  and  Eubanks  (1981). 

3.  F-15  Simulator  IOS  Operational  Test 
and  Evaluation.  An  operational  test  and 
evaluation-©?- an  F-15  flight  trainer  IOS 
was  conducted.  Measurements  of  the  IOS 
controls  and  displays,  workstation  dimen¬ 
sions,  and  workplace  environment  were 
compared  with  the  published  military 
standards  for  equipment  design.  These 
measurements  Included  CRT  display  character 
size,  pushbutton  control  size,  lighting 
levels,  ambient  temperature,  noise  levels, 
writing  surface  depth  and  height,  and  reach 
distances.  In  addition,  a  number  of  F-15 
IOS  users  were  Interviewed  to  determine 
operational  deficiencies  and  recommenda¬ 
tions  were  provided  for  correcting  these 
deficiencies.  This  evaluation  was  accom¬ 
plished  In  1977  for  the  Air  Force  Tactical 
Air  Warfare  Command  1TAWC). 

4.  F-5E  Instrument  FMght  Simulator 
Operational  test  and  Evaluation.  TF  T70T 


supported  an  operational  test  and  evaluation 
of  the  F-5E  Instrument  Flight  Simulator. 
This  effort  Involved  the  development  of  the 
test  procedures  and  data  collection 
materials  as  well  as  assistance  in  the 
collection  and  analysis  of  the  test  data. 
The  data  collection  materials  consisted  of 
rating  forms  which  were  administered  to 
F-5E  IPs  to  determine  the  fidelity  and 
training  capability  of  the  simulator 
cockpit  and  the  operability  and  instruc¬ 
tional  capability  of  the  IOS.  The  data 
collection  effort  Involved:  (1)  briefing 
the  pilots  and  IPs  on  the  missions  to  be 
performed;  (2)  distributing  the  rating 
forms;  (3)  monitoring  the  activities  of  the 
IPs  at  the  IOS  and  the  pilots  In  the 
cockpit;  and  (4)  recording  their  comments 
concerning  the  simulator.  The  evaluation 
was  managed  by  the  Air  Force  Test  and 
Evaluation  Center  (AFTEC). 

5.  A-10  Operational  Flight  Trainer 
Operational  Test  and  Evaluation^  TRe 
operational  test  and  evaluation  of  the  A-10 
Operational  Flight  Trainer  was  also 
supported  by  AFHRL/OT.  The  support  involved 
assistance  In  the  development  of  data 
collection  materials  and  collection  of  test 
data.  The  data  collection  materials 
consisted  of  rating  forms  that  were  admin¬ 
istered  to  the  test  IPs  to  assess  the 
fidelity  and  training  capabilities  of  the 
A-10  simulator  cockpit  and  the  instruc¬ 
tional  capability  of  the  A-10  IOS.  This 
evaluation  was  also  managed  by  AFTEC. 

6.  SAAC  IOS  Control  Panel  Design.  The 
control /indicator  panel  For  the  Aerl  al 
Combat  Engagement  Display  (ACED)  of  the 
Simulator  for  Alr-to-AIr  Combat  (SAAC)  was 
redesigned.  The  ACED  Is  essentially  a  CRT 
which  is  capable  of  displaying  several 
pages.  These  pages  present  a  view  of  the 
gaming  area  and  the  various  aircraft  in  the 
area,  a  pilot’s  view  from  Inside  the  cock¬ 
pit  of  the  visual  scene,  and  performance 
scoring  data.  Controls  and  Indicators  are 
used  to  display  different  pages  and  to 
change  range  scales.  In  the  modified  panel 
design,  the  ACED  controls/indicators  that 
were  originally  contained  in  two  separate 
panels  were  consolidated  Into  a  single 
panel.  The  controls/indicators  were 
arranged  In  functional  groups.  The  panel 
design  has  been  Implemented  on  the  SAAC. 

AFHRL/OT  IOS  RAD  IN  PROGRESS 

1.  IOS  Design  Guide.  A  guide  for 
flight  simulator  IOS  design  Is  currently 
under  development  by  AFHRL/OT.  The  guide 
will  be  made  up  of  three  sections,  or 
volumes.  The  first  volume  will  Include 
detailed  descriptions  of  advanced  flight 
simulator  IOS  configurations.  The  second 


will  contain  a  compilation  of  the  appro¬ 
priate  human  engineering  design  criteria. 
The  third  win  provide  an  IOS  design  for 
fighter/attack  flight  simulators  based  on 
the  data  In  the  first  two  vo’umes.  At 
present,  several  simulation  training 
facilities  have  been  visited  and  descrip¬ 
tions  of  the  IOSs  have  been  prepared. 

The  facilities  that  were  visited  include: 
M)  F-15  Instrument  Flight  Simulator-  (2) 
A-10  Operational  Flight  Trainer;  (31  F-14 
Weapons  System  Trainer;  (41  Tactical  Air¬ 
crew  Combat  Training  System  (TACTS1-  and 
(S)  EA-6B  Weapons  System  Trainer.  During 
these  visits  the  operation  of  several  other 
simulation  systems  was  observed,  namely, 
the  F-14  Operational  Flight  Trainer,  F-14 
Mission  Trainer,  TA-7J  Operational  Flight 
Trainer,  A-6E  Weapons  System  Trainer,  and 
A-6E  Might  Carrier  Landing  Trainer.  Tactics 
Information  will  not  be  included  In  the 
report  because  of  its  classified  nature. 
Descriptions  of  the  IOS  for  the  F-16,  F-18, 
and  F-4  trainers  will  also  be  Incor¬ 

porated  in  the  report.  The  second  volume 
will  contain  relevant  human  engineering 
design  criteria,  standards,  and  specifica¬ 
tions  obtained  from  a  compilation  of 
existing  data  and  from  studies  conducted  at 
AFHRL/OT.  ExampTes  of  the  areas  that  will 
be  addressed  in  the  IOS  design  guide 
IncTude  control /display  design  and  layout, 
workstation  configuration,  workplace 
environment,  anthropometry,  and  operator 
seating.  Volume  three  will  provide  a 

conceptual  IOS  design  specifically  for 
fighter /attack  simulator  applications.  It 
win  contain  detailed  design  specifications 
and  the  function  and  operation  of  the  IOS 
control  and  displays  will  be  explained. 
When  the  three- volume  report  Is  completed, 
sequel  will  be  prepared  for  tanker/ 
transport/bomber  simulators. 

2.  IOS  Interactive  CRT  Controls 
Eval  uatiofT  RIO  fias  been  Initiated  Tn 
which  alternative  control  devices  for  IOS 
Interactive  CRT  displays  will  be 
evaluated.  The  controls  are:  (1)  touch 
panel,  (2)  light  pen,  (3)  numeric  keyboard, 
and  (4)  voice  actuation.  These  controls 
will  be  compared  In  relation  to  three 
different  CRT  presented  performance  tasks. 
In  one  task,  the  subjects  will  be  asked  to 
load  various  weapons  from  a  menu  onto  the 
pylons  of  a  stylized  aircraft.  The  proce¬ 
dure  Is  to  select  an  appropriate  pylon 
number  with  the  controls,  then  select  and 
enter  the  weapon  number  Identifier.  For 
the  second  task,  the  subjects  will  be 
required  to  move  a  cursor  from  a  specific 
location  to  another  location  using  the 
controls,  which  Is  analogous  to  Initial¬ 
izing  a  simulated  aircraft  at  a  new  loca¬ 
tion  on  a  CRT  presented  IOS  navigation 
map.  The  third  task  will  require  the 


subjects  to  enter  numeric  data  to  establish 
the  simulator  parameters  such  as  altitude, 
airspeed,  heading,  radio  frequencies, 
latitude,  longitude,  and  glide  slope.  The 
subject  sample  will  consist  of  IPs  who  will 
be  tested  in  a  repeated  measures  experi¬ 
mental  design.  Both  response  times  and 
errors  will  be  recorded  to  provide  the 
measures  of  task  performance.  To  date,  the 
CRT  displays  (performance  tasks)  are  nearly 
complete  and  the  controls  are  operational. 
The  test  will  be  conducted  on  a  special- 
function^  part-task  trainer  which  is  com¬ 
prised  of  a  Chromatics  video  display 
terminal  and  a  North  Star  minicomputer. 

3.  ASPT  IOS  Energy  Maneuverability 
Displ ays~  Energy  Maneuverability  Displays 
(EMDs)  were  developed  for  the  IOS  of  the 
Advanced  Simulator  for  Pilot  Training 
(ASPT),  one  for  use  in  conjunction  with 
F-16  flight  training  and  the  other  for  A-10 
training.  These  displays  are  presented  on 
an  IOS  vector  graphics  CRT,  and  they  show 
the  current  energy  state  of  the  simulated 
aircraft.  Since  aircraft  maneuverability 
Is  a  function  of  available  energy,  these 
displays  will  enable  the  simulator  IP  to 
monitor  the  maneuvering  capability  of  the 
aircraft.  With  this  information,  the  IPs 
can  Instruct  their  student  pilots  on  how  to 
increase  energy  to  maximize  aircraft 
performance  without  exceeding  the  struc¬ 
tural  limits  of  the  aircraft.  An  experi¬ 
ment  will  be  conducted  to  assess  the 
utility  of  these  displays  for  flight 
Instruction  at  the  ASPT  IOS.  A  group  of 
F-16  and  A-10  IPs  will  be  asked  to  monitor 
the  corresponding  F-16  or  A-10  EMO  during  a 
series  of  preprogrammed  flight  maneuvers  in 
the  ASPT.  Following  the  demonstration,  the 
IPs  will  be  asked  to  rate  the  instructional 
utility  of  the  EMD  and  their  comments  will 
be  solicited  via  a  questionnaire.  The  EMDs 
implemented  in  the  proposed  study  were 
adapted  from  the  EMDs  used  on  the  Navy's 
Tactical  Aircrew  Combat  Training  System 
(TACTS)  which  were  originally  developed  by 
Pruitt  (1979). 

4.  IPS  Tactics  Display  Development. 
RAO  has  been  initiated  to  develop  an  IOS 
display  for  al  r-to-surface  tactical 
training.  A  conceptual  model  of  the  display 
has  been  designed  which  will  be  implemented 
and  evaluated  on  the  ASPT.  The  display 
provides  the  essential  aircraft  flight 
parameters  and  tactical  information  such  as 
altitude,  airspeed,  and  G-load  as  well  as 
an  overhead  view  of  the  tactical  area  with 
fixed  range  rings  to  show  the  range  and 
bearing  of  ground  radar  and  surface-to-air 
missile  launches.  The  simulated  aircraft 
Is  positioned  in  the  center  of  the  display 
and  the  terrain  moves  In  accordance  with 
the  speed  and  heading  of  the  aircraft.  In 
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the  evaluation,  a  group  of  IPs  will  be  asked 
to  Monitor  the  display  during  a  series  of 
preprogramed  tactical  Missions  In  which 
radar  will  lock  on  and  Missiles  will  be 
fired.  The  IPs  win  be  surveyed  to  deter- 
Mlne  the  utility  of  the  display  for 
Instructing/Monitoring  pilots  In  slMulated 
tactical  training  scenarios. 

5.  IPS  Integrated  Visual  01  splay 
OevelopminT  an  experimental  effort  Is 
underway  to  deve’op  an  IOS  Integrated 
visual  display  for  use  by  IPs  during  basic 
(T-37)  simulator  flight  training.  The 
purpose  of  the  display  Is  to  provide  all 
the  critical  flight  instrument  data  on  a 
single  CRT,  similar  In  form  to  a  head-up 
display  'HUD) .  In  this  effort,  a  question¬ 
naire  was  developed  and  administered  to  25 
T-37  IPs  to  determine  the  display  require¬ 
ments  for  Instructing/  monitoring  basic 
training  missions  Including  contact  flight, 
aerobatics,  formation  flight.  Instrument 
flight,  and  navigation.  The  questionnaire 
data  are  currently  being  analyzed  to 
identify  the  specific  flight  Information 
required  for  each  of  the  missions.  The 
appmprlate  d1sp1ay  symbology  will  be 
selected  to  depict  this  Information  and  a 
display  will  be  designed  containing  the 
symbo’ogy. 

6.  IOS  Instructional  Features  Require¬ 
ments.  A  survey  was  recently  Initiated  lo 
identify  the  utility  of  advanced  instruc¬ 
tional  features  in  aircrew  training  devices. 
The  survey  wll1  be  administered  to  IPs  at 
various  flight  simu’atlon  facilities.  The 
IPs  will  be  asked  to  rate  how  frequently 
the  IOS  advanced  Instructiona1  features  are 
used  In  flight  training,  how  easy  Is  It  to 
use  each  feature,  and  the  training  value 
and  training  potential  of  the  features.  A 
total  of  17  features  will  be  examined 
including  Freeze,  Record/Playback,  Reset, 
Demonstration,  Crash  Override,  Automated 
Performance  Feedback,  and  Automated  Adap¬ 
tive  Training.  The  survey  will  be  conducted 
at  A-10,  F-15,  and  F-4  simulator  training 
sites.  The  results  of  the  survey  will  be 
used  as  a  guide  for  specifying  the  required 
Instructiona1  features  In  future  flight 
simulator  IOS  designs. 


AFHRL/OT  has  provided  scientific  and 
engineering  support  to  a  variety  of  flight 
simulator  IOS  design  and  evaluation  efforts. 
Many  of  these  efforts  have  been  In  support 
of  outside  Air  Force  agencies  which  have 
involved  the  assessment  of  IOS  Instructional 
utility  and  the  deve’opment  of  generalized 
guidelines  for  IOS  design.  Other  efforts 
have  been  accomplished  strictly  in  house  to 
solve  specific  IOS  design  problems.  It  Is 
anticipated  that  AFHRL/OT  will  provide 
continued.  If  not  expanded,  IOS  RIO 
support.  Greater  emphasis  will  be  placed 
on  the  use  of  desk-top,  special  functions 
trainers  In  the  conduct  of  future  IOS 
design  studies.  These  devices  will  be  used 
to  investigate  IOS  control  and  display 
designs  as  building  blocks  for  the  develop¬ 
ment  of  full -miss  Ion  flight  trainers. 
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INTRODUCTION 

In  the  past,  training  devices,  as 
with  many  other  systems,  were  all  too 
often  "provided”  to  the  user  and  left 
for  him  to  use  as  best  he  could.  The 
user  neither  had  been  involved  in  the 
specification  nor  in  the  design  of  the 
device.  As  a  result,  the  operator  in¬ 
terface  or  instructor  operating  sta¬ 
tion  (IOS)  was  generally  designed  as  a 
simulation  control  unit  rather  than  as 
a  training  control  unit.  Fortunately, 
the  early  devices  simulated  relatively 
simple  systems  and  were  seldom  used 
for  full  system  or  crew  training. 

Thus,  the  instructor  with  the  help  of 
a  technician,  could  generally  "oper¬ 
ate"  the  device. 

Two  factors  have  significantly 
changed  this  picture.  First,  the 
application  of  advanced  digital  tech¬ 
nology  to  training  devices  has  expan¬ 
ded  their  capabilities  enormously. 
Existing  weapon  system  trainers  (WST) 
can  provide  multiple  targets  in  an 
electronic  and  missile  warfare  en¬ 
vironment  with  visual  and  motion  simu¬ 
lation.  Freeze,  dynamic  replay,  re¬ 
set  and  demonstration  functions  are 
generally  available.  Graphics  dis¬ 
plays  provide  the  instructor  with 
"pilot  view"  displays  and  plots.  Hun¬ 
dreds  of  pages  of  alphanumeric  data 
on  simulation  options  are  provided. 

The  second  factor  was  the  in¬ 
creased  emphasis  on  the  use  of  simu¬ 
lation  for  training  which  occurred  in 
the  1970s.  Rising  costs  of  weapon 
system  ownership  and  operation  led  to 
a  renewed  application  of  simulation 
to  crew  training. 


follow  the  pace  set  by  simulation 
technology.  Relatively  simple  con¬ 
soles  designed  primarily  to  control, 
the  simulation*  had  proven  acceptable 
for  the  early  devices,  even  though 
not  optimum.  In  the  absence  of  any 
more  refined  requirements,  the  con¬ 
soles  for  the  new  sophisticated  train¬ 
ers  have  followed  much  the  same  ap¬ 
proach.  Unfortunately,  the  conse¬ 
quences  have  produced  serious  opera¬ 
bility  problems  as  well  as  sub-opti¬ 
mum  training  support.  While  the  de¬ 
vices  can  come  close  to  duplicating 
the  weapon  system  characteristics, 
they  cannot  necessarily  be  operated 
to  train  effectively  using  that  capa¬ 
bility. 

The  Naval  Training  Equipment 
Center  undertook  a  series  of  studies 
in  the  1970s  to  look  at  instructor 
console  problems.  One  of  these  stud¬ 
ies  (Charles,  1975)  looked  at  the  role 
of  the  instructor  pilot  (IP)  in  simu¬ 
lator  training.  The  study  which  sur¬ 
veyed  all  of  the  aviation  readiness 
training  squadrons  and  the  related 
simulator  installations  found  that 
several  changes  had  occurred  in  the 
operation  and  use  of  the  devices.  Of 
more  importance  was  the  expressed 
concern  that  the  next  generation  of 
trainers  which  were  then  in  design 
and  development,  would  prove  inoper¬ 
able  by  instructor  personnel  if  the 
simulation  control  orientation  of  the 
IOS  design  continued.  The  extensive 
use  of  cathode  ray  tube  (CRT)  dis¬ 
plays  would  compound  the  problem. 

IOS  PROBLEM  AREAS 


Thus,  with  the  requirement  for  ex¬ 
panded  use  of  simulation  and  with  the 
technology  available  to  implement  the 
requirement,  a  wide  variety  of  train¬ 
ers  were  developed  and  installed. 

Each  major  weapon  system  training  site 
now  has  at  least  a  WST,  and  in  addi¬ 
tion,  most  have  part  task  trainers 
(PTT)  to  support  crew  position  train¬ 
ing. 

Unfortunately,  research  and  de¬ 
velopment  effort  in  IOS  design  did  not 


When  operability  problems  did 
arise  in  some  of  the  newer  WSTs,  the 
NAVTRAEQUIPCEN  undertook  a  review  of 
the  IOSs  with  the  goal  of  identifying 
both  the  causal  factors  and  feasible 
solutions.  Two  WST  IOSs  have  been  re¬ 
viewed,  the  EA-6B  WST  Device  2F119 
and  the  F-14A  WST  Device  2F112.  In 
general,  the  problems  which  were  iden¬ 
tified  are  similar  in  both  devices. 
They  reflect  two  basic  development 
problems,  the  lack  of  human  factors 
engineering  effort  during  design  and 
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development  and  the  failure  to  define 
the  operational  environment  and  re¬ 
quirement  for  the  device  in  sufficient 
detail. 

While  the  human  engineering  pro¬ 
blems  are  significant  and  interact 
with  the  operational  problems  to  com¬ 
pound  the  operability  problem,  the 
operational  problems  are  of  prime  im¬ 
portance.  The  best  human  engineered 
IOS  cannot  solve  the  user  or  oper¬ 
ational  problems  which  exist.  However 
conversely,  the  most  user  responsive 
system  will  also  be  ineffective  if  in¬ 
operable. 

Although  steps  have  been  and  are 
being  taken  to  solve  some  of  the  oper¬ 
ability  problems,  the  training  or  uti¬ 
lization  problems  in  general  remain. 
These  problems  are  centered  about 
three  areas : 


a. 

definition 

of 

the 

user 

b. 

definition 

of 

the 

use 

c. 

definition 

functions 

of 

the 

training 

d. 

definition 

of 

the 

operators 

While  the  three  are  clearly  in¬ 
terrelated,  each  has  a  unique  impact 
on  IOS  design  requirements  since  they 
identify  the  simulation  requirements, 
the  trainer  and  training  functional 
requirements  and  the  operator/instruc¬ 
tor  interface  requirements. 

USER  DEFINITION 

The  user  refers  to  the  units 
which  actually  utilize  the  trainer  in 
their  training  program.  In  the  U.S. 
Navy,  the  user  and  the  custodian  and 
maintainer  are  not  the  same  activity. 
Most  of  the  WSTs  are  supported  by  the 
Fleet  Aviation  Specialized  Operational 
Training  Group  (FASOTRAGRU  detachments. 
The  trainers  are  maintained  and  read¬ 
ied  for  training  as  scheduled  by 
FASOTRAGRU  personnel.  They  also  pro¬ 
vided  technical  support  to  the  oper¬ 
ation  of  the  device.  The  primary 
users  are  the  Fleet  Readiness  Squa¬ 
drons  (FRS)  and  the  fleet  squadrons. 
Their  requirements,  although  similar 
in  terms  of  the  weapon  system  and  its 
operation,  differ  in  terms  of  the 
training  program  content  and  implemen¬ 
tation. 


The  FRSs  conduct  several  levels 
of  a  syllabi  designed  to  familiarize 
the  replacement  aircrew  with  the 
weapon  system  and  its  operational 
missions.  The  majority  of  replace¬ 
ment  aircrew  are  recent  graduates  of 
the  Naval  Air  Training  Command  and  . 
are  designated  Naval  Aviators  and 
Naval  Flight  Officers.  They  can  be 
considered  as  the  bachelors  degree  of 
the  aviation  community  and  have  the 
basic  professional  qualifications. 

The  FRS  training  programs  are  de¬ 
signed  to  produce  aircrews  who  are 
qualified  to  operate  a  specific 
weapon  system  and  to  join  a  fleet 
squadron  for  additional  training  prior 
to  deploying.  Although  many  of  the 
training  programs  have  been  task  ana¬ 
lyzed  and  behavioral  and  training  ob¬ 
jectives  identified,  the  thrust  of 
the  typical  FRS  training  program  is 
not  criterion  based,  nor  is  training 
to  proficiency  the  primary  objective. 
The  FRS  has  been  likened  to  a  graduate 
school .  As  in  academic  graduate 
schools,  while  grades  are  utilized, 
they  are  considered  more  for  informa¬ 
tion  than  for  a  pass-fail  function. 

The  replacement  aircrew  are  qualified 
aviators  and  NFOs,  and  they  are  con¬ 
sidered  to  be  capable  of  transitioning 
to  the  new  system  in  the  allotted 
training  time.  When  unsatisfactory 
performance  does  occur,  it  is  initial¬ 
ly  handled  with  the  usual  options  of 
counseling,  extra  time  and  tutoring. 

In  summary,  the  FRS  training  pro¬ 
gram,  given  the  limited  training  time, 
limited  flight  time,  and  limited  simu¬ 
lator  time,  is  organized  to  give  the 
replacement  aircrew  the  maximum  train¬ 
ing  and  experience  possible  while 
closely  monitoring  performance  for 
safety  and  correct  operating  proce¬ 
dures. 

Fleet  squadron  training  is  simi¬ 
larly  conceived  and  except  for  NATOPS 
and  instrument  checks  which  have 
associated  performance  criteria,  the 
rest  of  the  training  is  aimed  at  im¬ 
parting  as  much  experience  and  know¬ 
ledge  across  the  mission  requirements 
as  possible  prior  to  deployment. 
Readiness  criteria  have  been  developed 
but  are,  in  general,  more  experience 
oriented  rather  than  performance  ori¬ 
ented. 

These  training  programs  depart 
from  the  classic  training  approach 
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which  is  organized  around  a  highly 
structured  syllabus,  testing  and  fixed 
criteria. 

In  summary,  the  discussion  of  the 
user  programs  has  emphasized  the  in¬ 
structor  oriented  nature  of  the  pro¬ 
gram  since  it  serves  to  frame  some  of 
the  operational  problems  in  existing 
IOS  design. 

USE  DEFINITION 

The  FRS  use  of  the  WST  varies  as 
a  function  of  the  phase  of  training. 

The  FRS  syllabus  begins  with  systems 
familiarization  and  progresses  through 
basic  flight  characteristics  to  weapon 
system  mission  and  tactics  training. 
Procedures  and  position  training  are 
conducted  on  PTTs  if  they  are  avail¬ 
able.  If  not,  the  WST  is  used  to 
support  all  phases  of  training  includ¬ 
ing: 

•  system  familiarization 

•  basic  flight 

•  operation  and  basic  tactics 

•  advanced  tactics 

•  special  events 

Procedures  and  basic  flight  train¬ 
ing,  when  conducted  on  the  WST,  gen¬ 
erally  utilize  a  manual  training  mode 
in  which  the  instructor  can  control 
the  initial  conditions  and  the  evolu¬ 
tion  of  the  subsequent  events.  The 
instructors  consider  it  essential  to 
control  the  onset  and  removal  of  mal¬ 
functions  and  emergencies.  They  also 
find  it  easier  and  more  realistic  to 
simulate  the  various  controller  func¬ 
tions  in  a  manual  mode. 

Systems  operations  and  basic 
tactics  training  is  generally  conduc¬ 
ted  in  a  semi-manual  mode  if  possible. 
Pre-programmed  initial  condition  sets 
are  used  for  establishing  the  target  (s) 
and  environment  as  well  as  the  flight 
starting  point.  The  instructor  then 
assumes  control  and  adjusts  the  sce¬ 
nario  to  meet  aircrew  performance  and 
training  needs. 

Advanced  tactics  and  battle 
problem  training  events  are  more  folly 
programmed  because  of  the  almost  im¬ 
possible  task  of  trying  to  control 
manually  the  targets  and  the  many 


variables  involved.  Some  instructor 
interaction  is,  however,  required  to 
reorganize  or  reorient  the  problem 
based  on  aircrew  actions. 

Special  events  such  as  spin,  high 
angle  of  attack  and  missile  defense 
training  are  generally  conducted  in 
the  manual  mode  since  they  involve 
"free  flight"  by  the  student. 

Finally,  although  not  yet  imple¬ 
mented,  programmed  and  standard 
mission  events  are  needed  to  permit 
crew  and  unit  evaluations.  Simula¬ 
tion  provides  a  feasible  means  of 
accomplishing  such  operations  since 
realistic  targets  and  weapons  (and 
weapons  effects)  are  generally  un¬ 
available  for  "shoot-offs". 

Fleet  squadron  use  is  similar  to 
FRS  use  except  that  familiarization 
training  (procedures  and  basic  flight) 
is  not  part  of  the  training  program. 
Detailed  training  event  guides  and 
grade  sheets  are  generally  not  utili¬ 
zed. 

In  summary,  the  use  of  the  WSTs 
in  FRS  and  fleet  squadron  training 
includes: 

•  manual  modes  for  familiariza¬ 
tion  training  (if  conducted  on  the 
device)  as  well  as  for  special  "ex¬ 
periential"  events  such  as  spins  and 
missile  defense. 

•  semi-manual  modes  for  systems 
and  basic  tactics  training 

•  programmed  or  formulated 
modes  for  advanced  tactics  and  battle 
or  war  problems 

TRAINING  FUNCTIONS  DEFINITION 

Effective  training  requires  a 
series  of  tasks  ranging  from  instruc¬ 
tor  preparation  for  the  event  to  re¬ 
cord  keeping.  The  set  of  tasks  used 
to  analyze  the  IOS  designs  included 
the  following: 

a.  Prepare  -  review  event,  air¬ 
crew  files,  simulator  status; 
get  forms,  guides,  manuals 

b.  Brief  -  review  event,  objec¬ 
tives  and  procedures  with 
aircrew  and  instructor  staff 
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c.  Initialize  -  configure  con¬ 
soles  and  cockpit;  select  and 
implement  initial  conditions 
or  programmed  mision 

d.  Train  -  instruct;  control  sim¬ 
ulation;  monitor  performance 

e.  Evaluate  -  evaluate  aircrew 
proficiency;  diagnose  perform¬ 
ance  problems 

f.  Debrief  -  review  event  results 
with  aircrew  and  instructors 

g.  Manage  data  -  update  aircrew, 
staff  and  simulator  files 

h.  Develop  events  -  create/pro¬ 
gram/modify  events,  displays, 
missions,  simulation  data 

i.  Train  instructor  -  train  in 
simulator  operation  and  use 

The  training  device  should  pro¬ 
vide  support  to  each  of  the  functions, 
especially  the  brief,  initialize, 
train,  evaluate,  debrief  and  develop 
functions. 

DEFINITION  OF  OPERATORS 

The  traditional  operators  of 
training  devices  have  consisted  of  the 
technician  simulator  operator  (SO) 
provided  by  the  FASOTRAGRU  Detachment, 
and  the  instructors  provided  by  the 
squadrons.  The  SO  is  typically  a 
technician  who  is  learning  console 
operation  prior  to  being  assigned  to 
a  maintenance  crew.  Thus,  in  general, 
the  SO  is  not  an  expert  in  simulator 
operation  although  such  assistance  is 
generally  available  on-call.  The  SO 
is  available  to  assist  the  instruc¬ 
tors  in  "button  smashing". 

The  instructors  from  the  FRS  will 
have  completed  the  instructor  training 
syllabus  which  generally  includes  a 
short  course  on  simulator  use  and  some 
on-the-job  experience.  Of  more  im¬ 
portance  is  the  fact  that  the  instruc¬ 
tor  will  also  be  instructing  on  other 
trainers  as  well  as  in-flight.  There¬ 
fore,  with  rare  exception,  he  is  not 
dedicated  to  WST  instructing.  During 
his  tour  of  duty,  he  will  also  be 
assigned  other  squadron  duties  and  his 
ground  instructing  time  will  taper  off 
as  a  function  of  time  and  duties  in 
the  squadron. 

The  fleet  squadron  instructors 


are  not  trained  in  simulator  operation 
(except  for  any  prior  FRS  experience) 
and  cannot  be  expected  to  become  qual¬ 
ified  in  operating  the  trainer. 

SUMMARY 

The  user  data  which  should  con¬ 
strain  the  IOS  design  involves  the 
specification  of  the; 

a.  user  -  primarily  FRS  and 
fleet  squadrons 

b.  use  -  primarily  system  oper¬ 
ation  and  tactics  (except 
when  supporting  familiariza¬ 
tion  training) 

c.  functions  -  all  training 
tasks 

d.  operator  -  the  technician 
operator  and  the  squadron  in¬ 
structors.  The  operator  is 
generally  new  to  the  device, 
the  FRS  instructor  has  lim¬ 
ited  training  in  operating 
the  device,  and  the  fleet 
squadron  instructor  will 
have  no  training  in  operating 
the  device. 

OPERATIONAL  PROBLEMS 

A  variety  of  problems  occur  in 
operating  typical  WSTs,  many  of  which 
seriously  impact  training  effective¬ 
ness.  Some  of  these  problems,  which 
result  from  failure  to  consider  the 
operational  environment  in  terms  of 
the  user,  the  use,  the  training  func¬ 
tion  support  and  the  operators  are 
outlined  below.  The  problems  will  be 
grouped  in  terms  of  IOS  configuration 
and  layout  problems,  display  and  con¬ 
trol  problems,  operating  problems  and 
training  function  problems. 

IOS  LAYOUT  PROBLEMS 

From  the  review  of  IOS  install¬ 
ations,  it  is  clear  that  insufficient 
attention  is  given  to  console  location 
and  layout.  In  general,  it  appeared 
that  the  typical  IOS  is  located  in  a 
main  traffic  flow  pattern,  and  be¬ 
comes  a  meeting  place  for  personnel 
in  the  area.  While  observation  of  on¬ 
going  training  can  contribute  to 
learning,  a  ready  room  environment  is 
not  conducive  to  either  the  training 
or  to  beneficial  observation.  Figure 
1  depicts  the  location  and  layout  of 
the  2F119  IOS.  All  traffic  to  and 
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from  the  cockpit  must  pass  through  the 
area.  In  addition  to  the  congestion 
problem,  the  aircrew  can  generally  get 
a  glimpse  of  the  upcoming  simulation 
being  initialized.  Light  spills  onto 
the  displays  when  doors  are  opened. 
Noise  interferes  with  inter-instructor 
communications . 

ARRANGEMENT  PROBLEMS 

Most  IOS  consoles  provide  a  sta¬ 
tion  for  each  of  the  instructors  as 
foreseen  by  the  specification  and  the 
design.  This  rarely  reflects  the  use 
and  user  needs.  One  of  the  major 
problems  occurs  in  the  battle  or  war 
problem  event,  especially  for  the 
fleet  squadrons.  As  can  be  seen  in 
Figure  1,  only  three  instructor  sta¬ 
tions  exist  (and  none  for  the  SO! ) 

The  battle  problem  instructor  who  is 
responsible  for  the  overall  evolution 
of  the  event  and  for  evaluation  and 
critiquing  of  crew  performance,  has  no 
effective  station  from  which  to  oper¬ 
ate.  The  IP  mans  the  flight  station; 
the  ECMO  instructor,  the  tactics  sta¬ 
tion.  In  practice,  the  battle  problem 
instructor  mans  the  ECMO  1  station 
where  he  is  forced  to  share  the  dis¬ 
plays  of  the  other  stations,  manipu¬ 
late  the  station  control  between  the 
flight  and  tactics  modes  of  operation 
and  perform  the  ECMO  1  instructor 
functions.  The  problems  are  similar 
for  the  2F112  as  can  be  seen  in  Figure 
2.  Here,  the  battle  problem  instruc¬ 
tor  is  forced  to  sit  behind  the  in¬ 
structor  stations  and  function  with  a 
clipboard  and  whatever  display  he  can 
read  from  this  position. 

DISPLAY  PROBLEMS 


The  CRT  display  provided  the  de¬ 
signers  the  opportunity  for  displaying 
literally  any  simulation  data  avail¬ 
able.  In  many  of  the  WSTs,  the  oppor¬ 
tunity  was  utilized.  The  volume  of 
pages  required  the  use  of  index  pages. 
Table  1  summarizes  the  display  options 
available  on  the  2F119  WST  flight  sta¬ 
tion  displays. 

The  quantity  of  data  far  exceeds 
the  capacity  of  any  IP  to  access  and 
utilize  effectively  during  training 
and  still  be  able  to  monitor  and  eval¬ 
uate  replacement  pilot  performance. 

The  options  cannot  be  accessed  by  an 
instructor  who  is  not  trained  and  pro¬ 
ficient  or  experienced  in  the  flight 
station  operation.  A  similar  set  of 
displays  exists  at  the  tactics  station. 


Figure  1.  Device  2F119  IOS  Arrangement 


TABLE  1.  2F119  FLIGHT  DISPLAY  OPTIONS 


Title 

Paqes 

Display  Index 

1 

Initial  Conditions  Index 

1 

Initial  Conditions 

10 

Pilot  Instrument  Monitor 

1 

Pilot  Console  Monitor 

1 

ECMO  1  Monitor 

1 

Procedures  Monitor  Index 

2 

Procedures  Monitor 

99 

Malfunctions  Index 

1 

Malfunctions 

13 

Imput  Codes 

2 

Function 

10 

Parameter  Recording 

1 

Cross  Country 

1 

Hostile  Environment 

1 

Terminal  Area 

1 

ACLS 

1 

GCA/CCA 

1 

CEM  Index 

1 

CEM  (Alphanumeric) 

variable 

CEM  (Graphics) 

1 

CEM  Summary 

1 

Demo  Index 

1 

Demos 

10 

DRED 

1 

Visual  Status  Monitor 

3 

Graphics  Test  Display 

2 

Memory  Monitor 

3 

System  Time 

5 

TOTAL 

167 

Figure  2.  Device  2F112  IOS  Arrangement 


A  similar  display  approach  was  im¬ 
plemented  on  the  2F112. 

CRT  cockpit  monitor  displays  on 
the  typical  console  pose  serious  prob¬ 
lems  since  the  data  is  not  displayed 
in  a  manner  readily  interpreted  by  the 
instructor  who  is  also  training  in 
flight.  The  displays  do  not  typically 
parallel  either  the  arrangement  or  the 
format  used  in  the  aircraft.  Figure  3 
depicts  a  typical  page  used  in  WSTs 
and  is  from  the  2F119.  As  can  be  seen, 
the  instructor  cannot  glance  at  the 
display  and  ascertain  switch  settings 
without  first  locating  the  control  by 
reading  the  list  and  then  reading  the 
control  setting. 

CONTROL  PROBLEMS 

The  typical  instructor  station 
has  two  CRT  displays  along  with  some 
cockpit  repeater  displays.  The  CRTs 
are  operated  by  selecting  the  CRT  (and 


sometimes  the  specific  area  on  the  CRT) 
to  be  utilized,  then  selecting  the 
display  mode  to  be  accessed  and  fi¬ 
nally  pagina  to  the  data  required. 

The  sequence  of  steps  which  involves 
both  switch  and  light  pen  operations, 
for  example,  is  time  consuming  and 
requires  the  instructors  concentrated 
attentions.  Errors,  which  often  occur, 
typically  require  repeating  the  entire 
sequence  of  steps. 

Light  pens  are  widely  used  for 
simulator  control.  Although  light 
pens  are  becoming  more  reliable,  they 
are  still  unacceptable  for  time  sen¬ 
sitive  inputs  such  as  malfunction  in¬ 
sertion,  weapons  launches  and  training 
control  functions.  In  the  past,  poor 
light  pen  reliability  has  severely 
handicapped  console  operations. 

The  typical  joystick  display- 
control  dynamics  involves  a  step  func¬ 
tion  which  renders  it  unusable  for  CRT 
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operations.  Manual  control  or  flying 
of  targets  using  the  joysticks  has 
proven  equally  impossible  because  of 
the  lack  of  sufficient  flight  infor¬ 
mation  and  because  of  joystick  control 
axes  coupling. 

Cockpit  configuration  and  initial 
condition  control  mismatches  generally 
require  extensive  manipulation  of  IC 
procedures  to  resolve  the  problem. 

Communication  options  controls  on 
most  WST  IOSs  have  proven  so  time  con¬ 
suming  to  use  and  so  error  producing 
that  the  options  are  not  utilized. 

The  control  panel  is  left  in  "over¬ 
ride"  . 

OPERATING  PROBLEMS 

Manual  modes  of  trainer  operation 
have  generally  proven  difficult  to 
utilize  even  though  required  for  many 
events.  The  exception  is  the  IC  which 
involves  starting  with  the  cockpit  pre¬ 
flight  procedure  at  the  take-off  end 
of  the  runway.  However,  even  for  this 
IC,  the  instructor,  unless  he  accesses 
relevent  data  pages  is  unsure  as  to 
fuel  state,  stores  configuration, 
weather,  etc.  The  major  problem  is 
one  of  knowing  on  what  page  and  in 
which  display  mode  the  relevent  data 
is  available,  and  when  accessed,  how 
to  edit  and  what  the  impact  will  be  on 
inter-related  parameters.  The  task  is 
beyond  the  "novice"  user. 

Programming  or  formulating  meth¬ 
ods  requires  extensive  training  and 
recent  experience  to  utilize.  This 
requirement  cannot  be  met  by  the 
typical  instructor.  The  procedures 
on  most  WSTs  do  not  provide  an  "inter¬ 
face"  between  the  instructor  and  the 
mission  programmer. 

Target  creation  is  normally  ac¬ 
complished  in  simulation  parameters 
rather  then  in  user  terminology.  Thus, 
targets  are  created  in  terms  of  "small 
fighter"  with  an  "IR"  missile  and  a 
spot  jammer,  for  example,  rather  than 
in  terms  of  a  Badger  or  Backfire  or 
Muffcob.  The  result  is  that  the  in¬ 
structor  does  not  generally  know  what 
the  displayed  target  represents  or 
how  to  evaluate  the  aircrews  tactics 
relative  to  the  target. 

TRAINING  FUNCTION  PROBLEMS 

Ideally,  a  training  device  should 
be  able  to  support  each  of  the  train¬ 


ing  tasks  involved  in  the  training 
mission.  Few  do.  To  the  extent  that 
they  do  not,  additional  workload  is 
placed  on  the  instructor  or  SO  and 
additional  training  may  be  required. 
Table  2  is  a  summary  of  three  WSTs 
support  to  the  training  functions  out¬ 
lined  earlier. 


TABLE  2.  TRAINING  FUNCTION  SUPPORT 


Function 

2F111 

WST 

2F112 

2F119 

Prepare 

none 

none 

none 

Brief 

none 

none 

none 

Initialize 

yes 

yes 

yes 

Train 

yes 

yes 

yes 

Evaluate 

none 

none 

some 

Debrief 

none 

none 

none 

Manage  Data 

none 

none 

none 

Develop  events 

yes 

yes 

yes 

Train  instruc. 

none 

none 

none 

As  can  be  seen,  the  typical  WSTs 
are  not  designed  to  support  training 
functions.  As  has  been  discussed  in 
the  previous  sections,  even  those 
tasks  in  Table  2,  which  are  recorded 
as  "yes",  present  marginal  support  to 
the  function.  Yet,  each  of  the  de¬ 
vices  has  data  stored  within  the  sys¬ 
tem  which  would  be  useful  for  the  in¬ 
structor  in: 

•  reviewing  the  scenario  and 
options  prior  to  briefing 
the  student 

•  reviewing  the  weapon  system 
operating  procedures  and  simu¬ 
lation  options 

•  briefing  the  aircrew  on  event 
procedures,  scenario  and 
objectives 

•  briefing  the  instructor  staff 
on  the  scenario  and  training 
procedures 

•  debriefing  the  aircrew  on  the 
results  of  the  training 

•  debriefing  the  instructors  on 
the  problems  and  changes  re¬ 
quired 

•  updating  training  records 

Communications  simulation  is 
performed  manually  in  most  trainer 
and  can  require  up  to  two  instructors 
full  time  for  some  battle  problems. 
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Simulation  of  controllers  and  provid¬ 
ing  relevent  background  communications, 
especially  for  sequential  battle  prob¬ 
lems,  is  almost  impossible. 

None  of  the  WSTs  reviewed  pror 
vide  the  capability  of  either  briefing 
or  debriefing  the  aircrew  (or  instruc¬ 
tor  staff)  without  utilizing  training 
time  and  the  instructor  console.  None 
of  the  WSTs  provide  the  additional 
displays  and  controls  required  or  the 
interface  required. 

Hard  copy  output  on  most  WSTs  is 
typically  so  slow  and  involved  that  it 
is  not  used.  Yet,  all  instructors 
agree,  that  hard  copy  is  desirable, 
especially  if  no  debriefing  displays 
are  available. 

Malfunction  options  far  exceed  a 
usable  set  on  most  WSTs  (typically  in 
the  hundreds).  In  addition,  informa¬ 
tion  on  the  simulation  characteristics 
of  the  malfunction  is  not  provided. 

As  a  result,  the  instructor  has  no  in¬ 
formation  on  the  different  cockpit  in¬ 
dications  and  relevant  procedures,  for 
example,  for  different  engine  fires  of 
flight  control  system  failures. 

SUMMARY 

Recent  reviews  of  the  IOSs  of 
several  WSTs  have  documented  a  variety 
of  operational  problems.  In  general, 
they  result  from  both  the  failure  to 
consider  user  requirements  or  to  de¬ 
sign  the  IOS  to  basic  human  factors 
engineering  criteria. 
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ABSTRACT 

The  real  world  is  defined  as  the  operational  training  environment. 
Instructional  support  features  (ISF)  are  simulator  hardware  and  software 
capabilities  that  allow  instructors  to  manipulate,  supplement  and  otherwise 
control  the  student's  learning  experiences  with  the  intent  of  promoting  the 
rate  at  which  skills  are  learned  and  maximizing  the  levels  of  skills  achieved. 
This  paper  sunmarizes  research  findings,  observations  and  experiences  dealing 
with  the  design  and  use  of  the  following  ISFs:  automated  demonstrations; 
record  and  replay;  programmable  and  manual  malfunction  control;  automated 
cueing  and  coaching;  automated  controllers;  computer  controlled  adversaries; 
and  quantitative  performance  measurement.  Definitions  of  the  features  are 
given,  followed  by  comments  on  their  design  and  use.  A  final  section  deals 
with  persistent  needs  for  improved  instructor  training  in  the  use  of  simula¬ 
tion  in  general  and  in  ISFs  in  particular. 


BACKGROUND 

The  increasing  complexity  of  modern 
weapon  systems,  combined  with  their  ever 
increasing  performance  capabilities,  have 
created  a  situation  in  which  the  performance 
capabilities  of  crewmembers  are  becoming  com- 
mensurately  more  critical.  As  crew  perfor¬ 
mance  capabilities  become  more  critical,  the 
training  of  those  capabilities  becomes  an 
increasingly  difficult  problem  in  that  less 
performance  variance  can  be  tolerated. 

Training  using  operational  equipment  Is 
becoming  increasingly  difficult.  Acquisi¬ 
tion,  operation  and  maintenance  costs  are  all 
high.  Skilled  maintenance  and  Instructional 
personnel  often  are  scarce.  Finally,  envi¬ 
ronmental  constraints  constantly  encroach 
upon  real  world  training  areas  and  Impact  on 
how  they  can  be  used. 

The  use  of  aircrew  training  devices 
(ATDs),  while  not  as  glamorous  as  Inflight 
training,  has  several  Inherent  advantages. 
ATDs  still  are  relatively  inexpensive  to 
procure,  operate  and  maintain,  at  least  In 
comparison  with  actual  equipment.  Conse¬ 
quences  of  operator  error  or  Inability  to 
appropriately  respond  to  normal  and  emergency 
requirements  are  significantly  less  In  a  sim¬ 
ulator.  Control  over  training  events  Is 
vastly  superior  to  that  available  In  the  air. 
Simulators  can  be  used  to  create  low  proba¬ 
bility  conditions  at  will  for  training  pur¬ 
poses,  training  events  easily  can  be  repeated 
until  necessary  operator  skill  levels  are 
achieved,  and  their  utilization  rates  are 
significantly  greater  than  actual  equipment. 


The  question  now  has  become  how  to  max¬ 
imize  the  training  utility  of  ATDs  rather 
than  whether  to  use  them  in  training.  With 
the  proper  design,  application  and  use  of 
Instructional  support  features  (ISFs)  in 
training  devices,  the  utility  and  advantages 
of  training  devices  over  actual  equipment 
can  become  even  greater.  The  training  sim¬ 
ulator  can  become  an  even  more  Integral  part 
of  the  real-world  operational  training  envi¬ 
ronment. 


INSTRUCTIONAL  SUPPORT  FEATURES 

ISFs  are  features  of  training  simula¬ 
tors  that  are  specifically  designed  to  facil¬ 
itate  the  instructional  process.  ISFs  are 
the  hardware  and  software  capabilities  that 
allow  Instructors  to  manipulate,  supplement 
and  otherwise  control  the  student's  learning 
experiences  with  the  Intent  of  promoting  the 
rate  at  which  skills  are  learned  and  maxi¬ 
mizing  the  levels  of  skills  achieved.  They 
are  designed  to  allow  control  of  instruction- 
ally  related  variables  such  as  rate  of  con¬ 
tent  presentation,  amount  of  content  pre¬ 
sented  in  a  single  block,  amount  and  distri¬ 
bution  of  practice,  types  of  tasks  practiced, 
knowledge  of  results,  and  measures  of  perfor¬ 
mance.  ISFs  are  the  capabilities  that  trans¬ 
form  a  simulator  from  simply  a  practice  de¬ 
vice  into  a  flexible  element  of  the  total 
training  system.  Common  instructional  sup¬ 
port  features  include  freeze,  record  and 
replay,  and  programmable  initializing  con¬ 
ditions. 


Early  ISF  Implementation  efforts  were 
frequently  viewed  as  compromising  the  hard 
won  fidelity  of  simulators.  While  it  cer¬ 
tainly  is  possible  to  implement  ISFs  in  ways 
that  will  compromise  the  fidelity  of  simula¬ 
tion,  lower  levels  of  fidelity  are  not  the 
necessary  consequence  of  introducing  ISFs 
into  a  training  simulator.  The  features  can 
be  implemented  in  ways  that  will  leave  the 
fidelity  of  the  simulator  intact  and  at  the 
same  time  enhance  the  training  effectiveness 
and  utility  of  the  device.  The  commonly 
assumed  justification  for  high  levels  of 
fidelity  is  to  enhance  training  effective¬ 
ness.  Instructional  support  features  are 
capable  of  still  further  enhancing  training 
effectiveness  and  efficiency.  Thus,  instruc¬ 
tional  support  features  can  be  regarded  as 
"fidelity  plus."  This  statement  assumes 
that  the  implementation  of  the  support  fea¬ 
tures  is  accomplished  in  a  professional 
manner  that  does  not  compromise  the  fidelity 
of  the  simulation  itself  (e.g.,  video  dis¬ 
play  terminals  used  for  instructional  feed¬ 
back  are  not  placed  in  middle  of  an  otherwise 
accurate  instrument  panel). 

This  paper  does  not  attempt  a  discussion 
of  each  ISF  available  with  current  technolo¬ 
gy.  Rather,  several  were  selected  on  the 
basis  of  difficulties  that  seem  to  persist  in 
their  design  and  use.  The  content  of  this 
paper  is  based  in  part  upon  the  rather  mea¬ 
ger  experimental  research  data  that  exist  on 
Instructional  support  features,  and  in  part 
upon  observations  of  training  and  research 
applications  of  various  instructional  fea¬ 
tures.  The  following  ISFs  are  addressed: 

Automated  demonstrations; 

Record  and  replay; 

Programmable  and  manual  malfunction 
control ; 

Automated  cueing  and  coaching; 

Automated  controllers; 

Computer  controlled  adversaries;  and 

Quantitative  performance  measurement. 

Definitions  of  the  features  are  given, 
followed  by  comments  on  their  design  and  use. 
A  final  section  deals  with  persistent  needs 
for  improved  instructor  training  in  the  use 
of  simulation  In  general  and  ISFs  In  partic¬ 
ular. 

Automated  Demonstrations 

Automated  demonstrations  (auto-demo) 
permit  the  standardized  presentation  of  a 
mission  segment  or  entire  simulated  flight. 
All  simulator  systems,  Including  the  visual 
scene  (if  present),  motion  system,  primary 
flight  controls  and  displays,  crew  communi¬ 
cations,  and  sensor  displays  respond  as 
though  skilled  aircrews  were  at  the  controls. 


Auto-demo  requires  substantial  two-way 
communication  between  ATD  controlling  soft¬ 
ware  and  hardware,  and  crew  station  controls 
and  displays.  Given  this  requirement, 
auto-demo  is  found  only  in  more  sophisticat¬ 
ed  computer  controlled  ATDs.  Technically, 
auto-demo  can  be  constructed  for  any  segment 
of  flight  (e.g.,  takeoff,  fly  straight  and 
level,  perform  aerobatic  or  air  combat  ma¬ 
neuvers,  deliver  weapons,  fly  a  standard 
approach,  and  land)  without  the  student  or 
instructor  operating  any  primary  controls. 

Two  instructional  values  commonly  are 
assumed  for  auto-demo.  First,  the  feature 
provides  a  performance  model  that  the  student 
can  observe,  analyze,  pattern  his  own  be¬ 
havior  pattern  after,  and  use  as  a  reference 
for  self-evaluation  in  subsequent  trials. 

The  student's  workload  is  considerably  re¬ 
duced  during  auto-demo,  providing  him  with 
a  better  opportunity  to  observe  relationship-, 
among  cues  and  system  responses.  Similarly, 
instructor  workload  is  considerably  reduced, 
allowing  a  better  opportunity  for  instruc¬ 
tional  interaction  with  the  student. 

A  second  value  is  that  automated  demon¬ 
strations  may  be  the  only  way  to  show  the 
student  what  is  expected  of  him  in  ATDs  that 
exactly  reproduce  crews tati on  physical  con¬ 
figuration.  When  the  crewstation  is  either 
single  seat  (e.g.,  F-15)  or  incorporates 
controls  and  displays  necessary  to  execute 
manual  demonstrations  at  only  one  crew  posi¬ 
tion  (e.g.,  F-4),  there  is  no  opportunity 
for  an  instructor  to  enter  the  crewstation 
and  do  a  hands-on  demonstration.  Therefore, 
the  only  remaining  avenue  for  a  demonstra¬ 
tion  is  by  means  of  auto-demo. 

In  an  examination  of  auto-demo  utili¬ 
zation,  Semple,  Cotton  and  Sullivan  (1981) 
concluded:  "The  training  value  of  an  auto¬ 
mated  demonstration  will  be  greatest  when 
the  cues,  responses  and  task  performance 
requirements  being  demonstrated  are  new  to 
the  student  (i.e.,  they  are  not  highly 
familiar  to  him)." 

Navy  research  with  the  A-7  Night  Carrier 
landing  Trainer  (NCLT)  has  shown  that  auto¬ 
demo  has  a  "significant"  instructional  value 
(Brictson  and  Burger,  1976).  However,  there 
were  no  data  quantifying  the  specific  con¬ 
tribution  that  the  feature  made  to  overall 
training  effectiveness  or  efficiency.  This 
application  does  represent  a  case,  however, 
where  an  instructor-flown  demonstration  was 
not  possible  because  the  ATD  is  a  single 
seat  configuration.  No  other  method  for 
demonstration  was  possible. 

The  only  known  experimental  study  to 
address  the  auto-demonstration  capability 
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was  reported  by  Hughes.  Hannan  and  Jones 
(1979).  This  study  attempted  to  address  the 
Instructional  Impact  of  both  the  auto-demo 
and  record  and  replay  Instructional  features. 
Subjects  Mere  divided  Into  three  groups  Mlth 
each  group  receiving  the  same  basic  instruc¬ 
tion  folloMed  by  either  extra  practice,  re¬ 
corded  replays  In  lieu  of  one  practice  trial 
In  each  block  of  trials,  or  an  auto-demo  In 
lieu  of  one  practice  trial  In  each  block  of 
trials.  The  results  Indicated  that  extra 
practice  Mas  most  Instructional ly  beneficial, 
folloMed  by  record  and  replay  and  auto-demo 
In  that  order.  The  general  liability  of  these 
conclusions  must  be  questioned,  hOMever,  in 
that  the  basic  instruction  for  all  three 
groups  Incorporated  auto-demo,  and  automated 
instruction  Mas  used  in  lieu  of  instructors. 
Given  a  real-Morld  Instructional  setting, 
these  results  might  not  hold-up. 

There  are  two  basic  methods  for  creating 
auto-demos.  One  is  to  use  an  ATD's  record 
and  replay  capability.  This  method  requires 
that  a  proficient  aircrew  member  fly  the 
necessary  profiles  in  the  desired  manner 
using  desired  techniques.  This  method  has 
the  advantage  that  highly  proficient  crew¬ 
members  usually  are  available  and  the  dis¬ 
advantage  that  it  is  difficult,  even  for 
highly  proficient  aircrew  members,  to  fly 
profiles  with  the  perfection  often  judged 
necessary  for  a  performance  model  for  the 
student.  Also,  the  lack  of  well  defined 
criteria  for  the  largest  percentage  of  fly¬ 
ing  tasks  makes  it  difficult  for  instructors 
to  agree  on  what  constitutes  an  acceptable 
demonstration. 

The  second  method  for  creating  an 
auto-demo  is  to  develop  customized  software 
for  each  demonstration.  This  approach  has 
the  advantage  that  mathematically  perfect 
demonstrations  can  be  created.  It  has  the 
distinct  disadvantage,  however,  of  requir¬ 
ing  computer  software  specialists  to  create 
new  demonstrations  or  to  modify  initial 
demonstrations.  Both  manufacturer  and  user 
personnel  view  the  direct  software  approach 
as  the  more  time  consuming  and  costly  of  the 
two  methods  available. 

The  need  for  "perfect"  demonstrations 
must  be  questioned  for  either  method  of 
development.  Demonstration  of  "typical" 
performance  may  be  more  appropriate  in  that 
no  one  flies  "perfect"  maneuvers  in  the  real 
world.  Appropriate  tactics,  techniques  and 
perhaps  even  typical  errors  and  performance 
problems  should  be  emphasized,  rather  than 
precise  flight  path  control. 

Record  and  Replay 

Record  and  replay  Is  the  capability  of 
simulators  to  record  relevant  system 


parameters  and  then  use  these  data  to  re¬ 
create  student's  performance.  Typically,  the 
last  five  continuous  minutes  of  performance 
are  recorded.  After  freezing  the  ATD,  the 
instructor  selects  the  point  in  time  during 
the  last  five  consecutive  minutes  that  he 
wants  replay  to  be  initiated.  All  events  of 
consequence  are  reproduced  during  replay. 
Including  the  visual  scene  (if  present), 
motion  system  operation,  primary  flight  con¬ 
trol  and  display  movements,  and  appropriate 
sensor  display  content.  The  ATD  performs  as 
though  the  student  was  again  flying  the  re¬ 
played  segment  exactly  as  it  had  been  flown 
before. 

Experience  has  shown  that  record  and 
replay  is  used  more  often  in  the  training  of 
tasks  that  are  new  to  the  student  and  are 
relatively  complex  (for  him).  In  general, 
the  use  of  record  and  replay  has  centered  on 
undergraduate  pilot  training  and  on  the 
training  of  new  (to  the  pilot)  and  relatively 
complex  advanced  flying  skills,  including 
night  carrier  landing  and  air  combat  maneu¬ 
vering. 

The  primary  instructional  benefit  at¬ 
tributed  to  record  and  replay  is  that  it 
provides  both  students  and  instructors  with 
an  objective  recreation  of  the  student's  per¬ 
formance  that  can  be  examined  for  problems, 
errors  and  their  causes.  In  short,  the 
student  is  provided  with  objective  knowledge 
of  results  and  the  instructor  with  concrete 
evidence  from  which  to  suggest  areas  or 
techniques  for  the  improvement  of  the  stu¬ 
dent's  performance.  Replay  seems  to  be  most 
useful  when  the  student  is  not  aware  that  he 
had  made  an  error  or  is  uncertain  of  the  pre¬ 
cise  cause  of  his  error  and  resulting  per¬ 
formance  problem.  This  situation  is  most 
apt  to  arise  when  complex  tasks  are  being 
initially  learned.  Following  interviews 
with  instructor  pilots  throughout  the  mili¬ 
tary,  Semple  et  al.  (1981)  concluded:  "The 
training  value  of  record  and  replay  will  be 
greatest  when  the  cues,  responses  and  task 
performance  requirements  being  learned  are 
new  to  the  student." 

Record  and  replay  is  currently  used 
almost  exclusively  for  pilot  and/or  copilot 
training.  Replay  could  be  used,  however,  in 
a  crew  training  environment  as  a  means  to 
evaluate  and  improve  crew  interaction. 

Voice  replay  also  would  be  required  in  this 
application  since  some  of  the  better  indica¬ 
tors  of  crew  interaction  are  crew  communica¬ 
tions.  This  potential  value  for  replay  de¬ 
pends  heavily  on  how  well  individual  crew¬ 
member  responsibilities  are  defined. 

The  record  and  replay  feature  requires 
substantial  two-way  communication  between 
ATD  controlling  software  and  hardware,  and 


crew  station  displays  and  controls.  Thus, 
the  feature  is  found  only  In  more  sophisti¬ 
cated,  computer  controlled  ATDs.  ATD  hard¬ 
ware  and  software  must  be  configured  to  in¬ 
duce  changes  in  the  operation  of  simulated 
subsystems  without  intervention  by  the  stu¬ 
dent  or  the  instructor. 

Several  factors  should  be  considered 
when  implementing  a  record  and  replay  fea¬ 
ture.  For  replay,  a  total  of  five  minutes 
appears  to  be  more  than  adequate.  In  most 
uses,  1.5  to  2.0  minutes  probably  would  be 
sufficient.  The  amount  of  replay  time  to 
be  incorporated  into  an  ATD  should  be  deter¬ 
mined  on  a  case  by  case  basis.  An  alterna¬ 
tive  to  specific  time  increments  also  should 
be  given  consideration.  A  reasonable  alter¬ 
native  would  be  to  provide  the  instructor 
with  a  cueing  control.  Replay  then  could 
be  started  at  a  determined  time  prior  to 
when  the  cueing  control  was  activated  (e.g., 
20  seconds,  or  whatever  would  be  reasonable 
for  the  particular  training  application). 

This  would  allow  instructors  to  relate  re¬ 
play  to  training  events  rather  than  clock 
time.  Voice  replay  should  receive  serious 
consideration  for  training  tasks  where  voice 
conmuni cation  is  a  significant  element  of 
total  task  performance.  Finally,  there  may 
be  training  and  user  acceptance  values  in 
being  able  to  deactivate  replay  part  way 
through,  thereby  allowing  the  student  to 
assume  manual  control  using  a  "fly  out" 
capability. 

Malfunction  Control 

Procedures  trainers,  part  task  trainers, 
operational  flight  trainers  and  full  mission 
simulators  generally  include  capabilities  to 
simulate  a  variety  of  malfunctions.  It  is 
widely  accepted  that  ATDs  provide  a  safe  and 
controlled  environment  for  training  re¬ 
sponses  to  malfunctions  and  emergencies.  In 
such  settings,  responses  to  single  and  mul¬ 
tiple  malfunctions  can  be  trained  and  prac¬ 
ticed  either  in  isolation  or  in  mission  con¬ 
texts.  The  number  of  malfunctions  that  in¬ 
structors  can  present  to  a  student  typically 
ranges  from  60  to  several  hundred.  The  num¬ 
ber  typically  used  ranges  from  20  to  50. 

The  most  common  method  of  inserting  and 
removing  simulated  malfunctions  is  manual 
operation  of  controls  by  the  instructor. 

Types  of  controls  now  in  use  include  dedi¬ 
cated  pushbuttons,  programmable  pushbuttons, 
alphanumeric  keyboards,  and  touch  panels. 

Another  alternative  is  automated  mal¬ 
function  insertion.  One  assumed  value  of 
automatic  malfunction  insertion  and  removal 
is  that  it  unburdens  the  instructor  from 
more  routine  tasks,  freeing  his  attention 
and  time  for  more  important  instructional 


activities.  A  second  value  involves  student 
self  practice.  Some  training  managements 
encourage  students  to  practice  whenever  ATD 
time  is  available,  even  if  instructors  can¬ 
not  be  present.  In  these  cases,  automatic 
malfunction  insertion  could  be  used  to  struc¬ 
ture  training  sessions. 

Little  creative  thought  has  been  applied 
to  meaningful  methods  for  automatically  in¬ 
serting  malfunctions.  Time  into  the  mis¬ 
sion  is  the  most  common  method  and  has  proved 
unworkable  because  mission  time  often  does 
not  correlate  with  mission  events.  For 
example,  ATD  clock  time  into  a  mission  typ¬ 
ically  does  not  take  freeze  into  account. 

If  the  ATD  is  frozen  so  the  instructor  can 
work  with  the  student,  the  malfunction  clock 
keeps  running.  As  a  result,  a  new  malfunc¬ 
tion  can  be  inserted  at  a  very  inappropriate 
time  in  the  mission.  The  acceptability  of 
this  circumstance  is  low,  both  instruction- 
ally  and  in  terms  of  user  acceptance. 

Semple,  Vreuls,  Cotton,  Durfee,  Hooks 
and  Butler  (1979)  developed  a  functional 
specification  for  a  simulator  instructor's 
console  which,  among  other  things,  incor¬ 
porated  automated  malfunction  insertion  and 
removal.  In  the  initial  concept,  the  in¬ 
structor  could  select  a  malfunction  and 
select  from  a  list  of  initiating  conditions 
for  that  malfunction.  The  malfunction  would 
be  inserted  when  the  initiating  conditions 
were  met  and  would  be  removed  following  the 
completion  of  correct  student  responses.  In 
an  experimental  prototype  of  the  system, 
automatic  malfunction  insertion  and  removal 
were  incorporated,  but  instructors  did  not 
have  a  choice  of  initiating  conditions. 

During  a  preliminary  test  of  the  system, 
instructors  did  not  respond  favorably  to  the 
automated  malfunction  scheme  (Semple,  1982). 
Their  strongest  complaint  was  lack  of  flex¬ 
ibility.  However,  instructor  training  in 
system  utilization  was  minimal,  and  it  is 
not  known  whether  their  comments  would  hold 
if  instructor  training  in  the  feature  had 
been  more  rigorous. 

There  are  four  general  issues  which 
should  be  considered  when  implementing  mal¬ 
function  insertion  options  in  an  ATD.  One 
is  the  amount  of  malfunction  cue  recognition 
which  should  be  trained  in  versus  out  of  the 
ATD.  Using  valuable  training  devices  to 
train  content,  which  would  be  equally  appro¬ 
priate  for  other  media,  is  inefficient  train¬ 
ing  design.  Ease  and  convenience  of  use  by 
instructors  and  operators  is  equally  impor¬ 
tant.  If  a  feature  is  difficult  or  trouble¬ 
some  in  use,  experience  shows  that  it  is 
likely  to  go  unused  (Semple,  et  al.,  1981). 
Memory  aids  should  be  designed  into  the 
system  to  remind  instructors  which  malfunc¬ 
tions  are  engaged  and  which  are  available. 


Finally,  with  automated  malfunction  Inser¬ 
tion,  programming  mu'- v.  allow  for  meaningful 
failures  relative  to  the  mission  rather  than 
to  an  arbitrary  time-line  and  Instructors 
must  be  able  to  override  automated  insertions 
and  removals. 

Automated  Cueing  and  Coaching 

Cueing  messages  alert  students, 
e.g.  "check  altitude."  Coaching  mes¬ 
sages  instruct,  e.g.  "return  to 
flight  level  150."  Automated  cueing  and 
coaching  are  not  common  in  present  ATDs. 

Their  assumed  instructional  values  center 
on  the  promptness  and  accuracy  of  guidance 
and  feedback  information  provided  to  the 
student,  and  the  unburdening  of  instructors 
through  automation.  Such  systems  also  can 
provide  feedback  and  guidance  to  students 
when  a  qualified  instructor  may  not  be  pres¬ 
ent,  as  in  "extra  time"  practice.  They 
require  good  automated  performance  measure¬ 
ment  to  determine  what  messages  should  be 
given  to  the  student  and  the  timing  of  their 
delivery.  Automated  cueing  and  coaching  sys¬ 
tems  can  be  disruptive  if  messages  occur  too 
frequently,  which  suggests  the  need  to  be 
able  to  deactivate  the  system,  and  further 
suggests  the  desirability  of  decision  logics 
designed  to  keep  the  frequency  of  cueing  and 
coaching  messages  within  acceptable  bounds  in 
relation  to  student  skill  levels. 

A  programmed  mission  scenario  typically 
is  required  so  that  desired  performance  is 
defined  clearly.  A  quantitative  performance 
measurement  (QPM)  system  and  additional  de¬ 
cision  logics  also  are  required  for  deter¬ 
mining  message  content  and  timing.  A  QPM 
capability  is  needed  to  sense  when  student 
performance  is  less  than  what  is  required  for 
the  task  he  is  practicing.  When  differences 
are  found,  system  logics  are  needed  to  iden¬ 
tify  the  appropriate  mess,  ge  content.  Typ¬ 
ically,  a  cueing  message  would  be  transmitted 
first  and  performance  monitoring  would  con¬ 
tinue.  If  the  performance  deficiency  was 
not  corrected,  the  appropriate  coaching  mes¬ 
sage  would  be  transmitted.  If  the  deficiency 
continued,  either  coaching  messages  could  be 
continued  or  the  instructor  alerted  so  that 
he  could  intervene. 

Three  technologies  are  available  for 
creating  the  messages  to  be  transmitted  to 
the  student:  audio  tapes;  digitally  stored 
speech;  and  computer  generated  speech.  Com¬ 
puter  generated  speech  is  relatively  new  but 
Is  readily  understandable.  Digitally  stored 
speech  is  even  more  "human."  Taped  messages 
often  Involve  prolonged  search  times  and 
mechanical  unreliabilities.  Digitally  stored 
or  computer  generated  speech  technologies  are 
well  suited  to  automated  cueing  and  coaching 
message  delivery  because  the  messages 


involved  usually  are  brief,  and  computer 
memory  requirements  are  well  within  reason. 
Also,  changes  in  message  content  are  easily 
accomplished. 

The  issue  of  which  messages  should  be 
built  into  an  automated  cueing  and  coaching 
system  must  be  addressed  on  a  case  by  case 
basis.  The  analysis  should  begin  by  identi¬ 
fying  typical  performance  problems  that  go 
unattended  by  the  student.  Commonalities 
among  the  performance  problems  (and  associa¬ 
tion  cueing  and  coaching  messages)  then 
should  be  identified  so  that  the  smallest 
possible  set  of  cueing  and  coaching  messages 
can  be  identified.  Draft  messages  then 
should  be  developed  and  reviewed  for  clarity 
and  brevity.  Finally,  the  automated  cueing 
and  coaching  system  should  be  tried  out  in 
a  representative  target  operational  training 
setting  before  its  design  is  finalized. 

Present  applications  of  automated 
cueing  and  coaching  center  on  basic  flight 
and  navigation  task  training.  In  the  future, 
it  may  be  possible  to  incorporate  these  capa¬ 
bilities  into  procedures  task  training.  In 
one  application,  use  of  the  feature  varied 
considerably  from  instructor  to  instructor, 
which  was  to  be  expected  because  of  lack 
of  instructor  training  on  potential  values 
and  limits.  Some  instructors  indicated  that 
the  feature  was  used  "quite  often"  by  stu¬ 
dents  who  came  in  to  practice  on  their  own, 
but  they  could  not  quantify  the  amount  of 
use. 

It  is  likely  that  the  use  of  looser  per¬ 
formance  tolerances  is  desirable  early  in 
training  to  trigger  cueing  or  coaching  mes¬ 
sages.  This  could  serve  to  reduce  the  num¬ 
ber  of  messages  transmitted  to  the  student 
during  early  training  when  his  abilities  to 
perform  may  be  significantly  less  than  at  the 
conclusion  of  training,  and  when  distractions 
may  be  of  negative  training  value.  However, 
there  presently  are  no  guidelines  for  deter¬ 
mining  these  tolerances.  Further  research 
is  needed. 

Automated  Controllers 

Control  means  to  regulate  or  direct. 
Instructors  often  play  the  roles  of  air 
traffic  controllers,  tactical  controllers, 
or  they  control  the  actions  of  simulated  air¬ 
borne  threats.  The  automated  controller 
instructional  support  feature  is  designed  to 
assist  ATD  Instructors  in  providing  the  con¬ 
troller  function. 

Automated  controller  systems  incorporate 
models  of  the  specific  operational  situations 
that  they  control  and  require  automated  per¬ 
formance  measurement  capabilities  to  relate 
actual  student  performance  to  desired 
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performance.  Controller  messages  are  then 
appropriate  to  both  the  original  incoming 
message  and  the  operational  situation.  The 
combination  of  automated  speech  understand¬ 
ing,  situation  recognition  and  computer  gen¬ 
erated  speech  are  becoming  powerful  instruc¬ 
tional  tools  for  automated  controller  appre¬ 
ciations.  A  typical  example  involving  both 
speech  understanding  and  speech  synthesis 
might  be: 

Incoming  communication: 

From  Pilot 

"Approach  Control  -  X  RAY  1 
turning  to  final" 

Situation: 

Aircraft  X  RAY  1  is  turning 
onto  the  final  ILS  approach 
at  the  correct  altitude. 

Environmental  conditions  are 
those  selected  for  the  exercise. 

Outgoing  Communication: 

From  Automated  Controller 

"X  RAY  1  you  are  cleared  to 
land.  Wind  now  150  at  20 
gusting  27" 

Synthetic  voice-based  controllers  are 
expected  to  be  used  increasingly  for  many 
ATD  voice  applications  with  highly  struc¬ 
tured  vocabular ies. 

Potential  training  benefits  steming 
from  automated  controllers  lie  in  four  areas: 
1)  unburdening  instructors  and/or  ATD  oper¬ 
ators  from  playing  controller  roles;  2)  in¬ 
creasing  the  timeliness  and  correctness  of 
controller  messages  and  feedback;  3)  unbur¬ 
dening  instructors  from  the  measurement  of 
verbal  task  performance  (and  associated 
record  keeping);  and  4)  providing  a  new 
medium  through  which  students  and  ATDs  can 
interact  in  a  highly  natural  manner.  As 
examples  of  the  fourth  point,  it  is  tech¬ 
nically  possible  for  the  student  to  ask  the 
ATD,  "How  did  I  do  on  that  bomb  run?"  If 
the  system  has  the  necessary  performance 
models  and  performance  measurement  capabili¬ 
ties,  it  could  respond:  "Very  well,"  and 
provide  a  detailed  performance  diagnosis  if 
desired.  Automated  voice  technology  also 
opens  opportunities  for  very  precise,  auto¬ 
mated  student  coaching  and  cueing. 

Current  training  systems  which  incor¬ 
porate  computer  speech  understanding  are 
limited  to  Individual  word  recognition  (IWR) 
technology  to  Interpret  individual  words  or 
very  short  phrases.  This  technology  re¬ 
quires  very  precise,  stylized  speech  by  the 
human  and  requires  each  student  to  repeat 


each  phrase  or  word  up  to  10  times  to  "train" 
the  computer  to  understand  what  was  said. 

In  a  recent  prototype  training  system  evalua¬ 
tion,  recognition  rates  50  to  97%  were  found 
with  an  average  of  85%  (McCauley  and  Semple, 
1980).  This  is  far  below  the  95  to  99% 
recognition  rates  possible  under  ideal  con¬ 
ditions  (Lea,  1980).  Connected  speech  recog¬ 
nition  technology  has  recently  surfaced,  al¬ 
lowing  people  to  speak  more  naturally,  with¬ 
out  the  stylization  constraints  required  by 
IWR.  Also,  connected  speech  systems  seem 
to  be  easier  to  “train." 

With  respect  to  computer  generated 
speech,  present  technology  is  quite  adequate 
for  creating  words  and  sentences  that  can  be 
understood  by  the  human.  Work  continues  on 
ways  to  make  the  computer  generated  speech 
sound  more  natural.  Finally,  much  of  the 
technology  needed  to  create  the  mathematical 
models  and  performance  assessment  capabili¬ 
ties  required  by  automated  controllers  also 
exists.  However,  it  still  is  the  case  that 
all  such  models  require  experimental  testing 
and  fine  tuning. 

The  design  of  automated  controller 
models  involves  two  principal  considerations. 
First,  the  technology  of  computer  speech 
understanding  is  improving  very  rapidly. 
Computer  speech  technology  developments  in 
the  last  seven  years  have  emphasized  "appli¬ 
cations"  rather  than  the  development  of 
basic  principles  of  speech  understanding  and 
synthesis.  This  has  resulted  in  certain  sys¬ 
tem  inadequacies  at  this  time,  but  dramatic 
improvements  are  currently  under  development 
and  will  become  available  in  the  near  future 
(Cotton  and  McCauley,  1982). 

A  second  and  important  consideration  is 
the  design  of  the  operational  performance 
model  that  drives  the  controller.  Early 
controller  models,  for  example,  were  derived 
from  "text  book"  procedures  for  performing 
the  maneuvers  that  were  being  controlled. 

In  the  operational  world,  pilots  seldom  fly 
profiles  strictly  according  to  procedures. 
Human  controllers  are  aware  of  this  and  re¬ 
spond  accordingly.  For  example,  a  pilot  may 
choose  to  turn  to  intercept  his  final  ap¬ 
proach  course  at  a  distance  from  touchdown 
and  at  an  altitude  that  an  automated  con¬ 
troller  has  not  been  programmed  to  recognize 
as  the  initial  point  for  a  final  approach  to 
landing.  Two  things  can  result.  One  is 
that  the  controller  model  may  issue  spurious 
commands  because  it  has  not  correctly  recog¬ 
nized  initial  conditions  for  the  start  of 
the  approach.  Second,  the  automated  per¬ 
formance  measurement  system  that  provides 
information  to  the  controller  model  also  may 
be  "fooled"  because  of  a  departure  from  the 
procedure  it  has  been  programmed  to  accept 
as  baseline  performance.  This  can  provide 
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the  controller  model  with  inaccurate  infor¬ 
mation  and  can  result  in  a  poorer  score  on 
the  approach  than  actually  was  earned. 

An  automated  ground  controlled  approach 
(GCA)  controller  has  been  successfully  ap¬ 
plied  to  an  F-4E  simulation  as  an  integral 
part  of  the  Automatic  Flight  Training  Sys¬ 
tem  (AFTS)  (Swink,  Smith,  Butler,  Futas  and 
Langford,  1975).  This  instructional  appli¬ 
cation  of  automated  speech  technology  marked 
the  beginning  of  a  new  era  of  automated  con¬ 
trollers  for  ATOs.  While  the  F-4E  AFTS  GCA 
controller  required  a  resident  general  pur¬ 
pose  computer  and  a  disk  memory  system  to 
provide  a  limited  repertoire  of  GCA  oriented 
words  and  phrases,  modern  microelectronics 
technology  now  offers  similar  capability  on 
2  to  4  chips  with  repertoires  of  up  to  200 
words.  The  AFTS  technology  was,  however, 
found  sufficient  for  it's  purpose  and  sub¬ 
sequently  employed  in  A7  Air  National  Guard 
and  Greek  Air  Force  training. 

There  are  many  prototype  systems  either 
under  development  or  in  testing  which  incor¬ 
porate  automated  controllers  including  the 
Navy's  F-14A  operational  flight  trainer 
(Semple  et  al . ,  1979),  the  Precision 
Approach  Radar  Training  System  (PARTS) 
(McCauley  and  Semple,  1980)  and  an  Air 
Intercept  Controller  (AIC)  training  system 
(McCauley,  Root  and  Muckier,  1982). 

The  use  of  controller  models  in  air¬ 
crew  training  is  relatively  new.  The  tech¬ 
nology  and  the  "lessons  learned"  data  bases 
require  expansion.  It  is  strongly  recom¬ 
mended,  therefore,  that  all  automated  con¬ 
troller  models  be  evaluated  and  refined 
during  the  development  process  to  ensure 
that  the  models  function  accurately  before 
their  design  is  frozen. 

Computer  Controlled  Adversaries 

Computer  controlled  adversaries  fre¬ 
quently  are  referred  to  as  "iron  pilots." 
They  are  computer  models  that  control  the 
actions  of  simulated  adversary  aircraft. 

Iron  pilots  have  been  used  in  visually 
equipped  air  combat  simulators  such  as  the 
Air  Force  SAAC.  Northrop  Corporation's 
LAS/WAVS,  and  NASA's  Differential  Maneuver¬ 
ing  Simulators.  Properly  designed,  they  can 
provide  realistic  adversary  maneuvering 
while  unburdening  the  instructor  from  con¬ 
trolling  the  adversary.  When  combined  with 
an  automated  performance  measurement  capa¬ 
bility,  summary  information  can  be  generated 
describing  engagement  final  outcome,  of fen- 
si  ve/defensive  times  for  each  aircraft, 
time  in  the  gun  envelope,  time  in  missile 
envelopes  and  similar  performance  infor¬ 
mation. 


Three  primary  instructional  values  are 
associated  with  computer  controlled  adver¬ 
saries.  One  is  the  repeatability  of  the  be¬ 
havior  of  the  adversary,  which  is  consistent 
and  predictable  (by  instructors)  during 
training  and  provides  a  more  consistent  base¬ 
line  against  which  to  evaluate  student  per¬ 
formance  and  diagnose  learning  problems.  A 
second  value  is  the  unburdening  of  the  in¬ 
structor  during  training  so  he  can  concen¬ 
trate  on  student  performance  and  provide 
more  meaningful  and  timely  guidance  and  feed¬ 
back.  A  third  value  is  the  lessening  of 
specialized  instructor  skills  that  must  be 
developed  to  continuously  control  a  simu¬ 
lated  adversary  aircraft  from  a  remote  con¬ 
sole. 

Iron  pilots  with  selectable  levels  of 
pilot  skill  hold  considerable  potential  for 
air  combat  training.  Easier  adversaries 
could  be  used  earlier  in  training.  As  the 
student's  skill  levels  increase,  more  dif¬ 
ficult  adversary  reactions  could  be  selected. 
The  progressive  approach  to  adversary  capa¬ 
bilities  could  hasten  the  learning  process. 
This  was  the  rationale  behind  the  "normal" 
and  "difficult"  autopilot  adversaries  de¬ 
veloped  for  the  Northrop  LAS/WAVS  simula¬ 
tion.  Instructors  in  LAS/WAVS  used  both 
difficulty  levels  for  the  training  of  tran¬ 
sitioning  students  (Spring,  1976;  Payne, 
et  al. ,  1976). 

Experience  gained  with  iron  pilots  in 
the  SAAC  device  and  the  Northrop  LAS/WAVS 
suggests  three  relevant  design  considera¬ 
tions.  The  first  is  that  adversary  actions 
controlled  by  iron  pilot  models  must  be 
realistic.  Original  iron  pilots  were  "too 
good"  and  consequently  were  unrealistic. 

They  operated  on  perfect  information,  and 
their  decisions  were  made  almost  instantly. 
This  made  them  virtually  unbeatable.  They 
had  little  training  values  as  a  result. 

When  iron  pilots  are  designed  to  provide 
realistic  maneuvering,  they  are  well  re¬ 
ceived  by  instructors  and  are  used  exten¬ 
sively.  ,  The  second  consideration  is  that 
iron  pilots  with  selectable  "skill  levels" 
should  be  developed  so  that  adversary  re¬ 
sponses  can  coincide  with  student  pilot 
skill  levels  during  training.  The  third  con¬ 
sideration  is  to  incorporate  fundamental  or 
basic  adversary  maneuvering  capabilities 
(such  as  simple  turning  maneuvers)  for  use 
early  in  basic  air  combat  training. 

Quantitative  Performance  Measurement 

Quantitative  performance  measurement 
(QPM)  for  training  is  the  computer-based 
monitoring,  recording,  processing  and  dis¬ 
playing  of  objective,  quantitative  infor¬ 
mation  for  describing  and  diagnosing  student 
performance.  QPM  systems  have  been  fairly 
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common  In  research  simulators  for  over  ten 
years,  but  research  systems  are  not  well 
suited  for  operational  training.  They  have 
been  tailored  for  research  use  and  frequently 
produce  voluminous  performance  data  that  re¬ 
quire  subsequent  statistical  processing.  A 
QPM  system  designed  for  use  in  training  must 
perform  all  statistical  and  other  processing 
of  performance  data  in  real  or  near- real  time 
so  that  students  and  instructors  are  provided 
with  useful,  concise  and  timely  performance 
feedback  information. 

Practically  all  quantitative  measurement 
capabilities  in  existing  ATDs  or  ATDs  soon  to 
be  delivered  are  best  described  as  perfor¬ 
mance  monitoring  and  data  recording  capabili¬ 
ties.  They  allow  instructors  to  select  tol¬ 
erance  bands  (e.g.,  +/-  100  feet)  around 
various  performance  parameters  (e.g.,  alti¬ 
tude).  The  system  then  monitors  for  cases 
that  exceed  tolerance  band  values,  and  re¬ 
cords  and/or  reports  out  of  tolerance  condi¬ 
tions.  Such  limited  performance  monitoring 
capabilities  have  been  applied  effectively 
to  drive  automated  cueing  and  coaching  sys¬ 
tems  where  individual  parameter  variations 
have  been  assumed  to  have  instructional 
meaning.  However,  such  capabilities  have 
found  little  acceptance  for  performance  eval¬ 
uation  and  learning  problem  diagnosis  in 
day-to-day  training.  In  other  words,  such 
systems  are  not  used  by  instructors.  The 
volume  of  data  produced  by  such  systems  often 
is  overwhelming  and  is  difficult  to  integrate 
and  interpret.  Also,  instructors  almost 
never  are  trained  to  use  individual  parameter 
variation  data  meaningfully  for  training. 

APM  research  has  shown  that  some  mea¬ 
sures  contribute  more  to  the  total  descrip¬ 
tion  of  student  performance  than  do  others. 
Research  also  has  shown  that  individual  mea¬ 
sures  may  not  be  useful  for  discriminating 
between  "good"  and  "poor"  performance,  but 
properly  weighted  and  combined  are  quite  use¬ 
ful  for  discriminating  performance  differ¬ 
ences  (Waag,  Eddowes,  Fuller  and  Fuller, 
1975).  The  extent  to  which  various  measures 
must  be  weighted  and/or  combined  remains  a 
research  issue,  but  the  need  to  do  so  has 
been  demonstrated  for  basic  instrument  flight 
maneuver  training  (Vreuls,  Wooldridge, 
Obermayer,  Johnson,  and  Goldstein,  1976)  and 
air  combat  maneuvering  training  (Kelly, 
Wooldridge,  Hennessy,  Vreuls,  Barnebey  and 
Cotton,  1979). 

Quantitative  measurement  of  performance 
of  procedural  sequences  is  a  relatively  new 
technology.  A  number  of  newly  acquired  ATDs 
will  incorporate  procedure  monitoring  capa¬ 
bilities  (e.g.,  F-16,  A- 10,  F-5,  B- 52  and 
C-130).  These  systems  will  display  the  se¬ 
quences  in  which  procedures  are  performed. 

It  will  be  the  instructor's  responsibility 


to  determine  whether  or  not  the  performance 
is  acceptable. 

Under  Navy  sponsorship,  blends  of 
"manual"  and  "quantitative/automated'1  per¬ 
formance  measurement  capabilities  were  in¬ 
corporated  into  a  recent  experimental 
prototype  instructional  support  system 
(Semple  el  al.,  1979).  Among  other  factors, 
procedural  performance  was  displayed  on  a 
video  display  terminal.  An  ideal  sequence 
of  procedural  steps  was  displayed,  irrele¬ 
vant  procedures  were  separately  displayed 
as  they  occurred,  and  the  clock  times  at 
which  all  procedures  actually  were  performed 
were  displayed  beside  the  steps.  In  a 
rather  limited  initial  trial  use,  instruc¬ 
tor  pilots  found  this  display  valuable,  al¬ 
though  actual  event  times  were  not  consi¬ 
dered  necessary  (Semple,  1982). 

In  the  same  system,  procedural  perfor¬ 
mance  scores  were  derived  and  displayed. 

The  scores  were  based  on  algorithms  developed 
by  a  group  of  highly  qualified  instructor 
pilots,  and  following  much  heated  debate. 

In  practice,  the  weighted  scores  were  judged 
invalid  by  instructor  pilots.  The  lesson 
seems  to  be  that  valid  quantitative  per¬ 
formance  measures  (individually)  and  scores 
(collections  of  weighted  measures)  must  be 
derived  through  statistical  analyses,  at 
least  for  flying  training. 

Taken  together,  flying  training  QPM 
capabilities  which  emphasize  either  quanti¬ 
ties  of  individual  measures  or  analytically- 
derived  weightings  of  several  measures  will 
be  of  little  practical  value  until  both 
instructor  training  and  measurement  method¬ 
ologies  improve  with  respect  to  quantitative 
measure  indices. 

Guidelines  for  the  design  of  practical, 
valid  and  acceptable  QPM  systems  are  not  yet 
at  hand.  Human  performance  is  complex,  and 
one  human's  evaluation  of  another  human's 
performance  is  more  complex.  Computer-based 
systems  for  assessing  and  diagnosing  human 
performance  are  beginning  to  evolve,  but 
operational  applications  of  true  QPM  systems 
basically  are  non-existent  in  flying  train¬ 
ing.  Further  research  is  required. 


INSTRUCTOR  TRAINING 

Virtually  all  instructors  who  train 
other  pilots  using  aircrew  training  devices, 
other  than  basic  procedures  trainers,  are 
rated  airmen.  Typically  they  are  motivated 
and  dedicated  personnel  who  are  highly  com¬ 
petent  at  performing  the  tasks  they  are 
teaching  others  to  do.  They  may  have  been 
assigned  their  instructional  duties,  or 
they  may  have  volunteered  for  the  job. 
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Instruction  (In  ATOs  and/or  aircraft)  may  be 
their  primary  job  assignment,  or  It  may  be  an 
assignment  dominated  by  collateral  duties. 
Specific  training  for  their  instructional 
assignment  may  have  been  systematic  and 
rigorous,  but  more  likely  it  was  not. 

The  best  of  training  equipment,  by  it¬ 
self,  will  not  produce  operationally  ready 
aircrews.  The  equipment  must  be  used  effec¬ 
tively  and  efficiently  to  achieve  this  goal. 
Obviously,  instructor  training  should  be  cen¬ 
tral  to  the  effective  and  efficient  use  of 
ATDs.  This  comment  continues  to  have  con¬ 
siderable  face  validity  even  though  there  is 
no  empirical  evidence  indicating  that  rated 
personnel  are  required  for  flight-oriented 
training,  or  that  instructor  training  in 
instructional  principles,  ATO  use,  or  in¬ 
structional  feature  utilization  actually  has 
any  benefit  (Caro,  Shelnutt  and  Spears,  1982). 
The  fact  of  the  matter  Is,  these  Issues  never 
have  been  systematically  examined. 

It  seems  self  evident  that  ATD  instruc¬ 
tors  are  central  to  the  proper  use  of  train¬ 
ing  devices.  Part  of  the  Instructor's  job 
is  knowing  how  to  operate  ATOs.  A  second 
part  is  knowing  how  to  use  such  devices  and 
their  capabilities  effectively.  Achieving 
the  second  goal  requires  knowledge  of  device 
capabilities  along  with  principles  of  In¬ 
struction.  Present,  typical  military  In¬ 
structor  tralnln  rovldes  neither  with 
certainty. 

There  are  exceptions.  However,  typical 
military  pilot  Instructor  training  programs 
focus  on  how  to  perform  the  tasks  to  be 
trained,  safety,  and  training- related  ad¬ 
ministrative  matters.  On  average,  only  about 
three  hours  of  formal  Instruction  deals  with 
how  to  be  a  teacher.  The  operation  and  use 
of  ATDs  typically  is  left  to  informal  on-the- 
job  training.  Overall,  there  Is  little  In¬ 
structional  quality  control,  except  for 
standardization  and  evaluation  functions, 
which  may  or  may  not  focus  on  Instructional 
processes  and  products. 

There  is  no  question  that  the  military 
pilot  training  system.  Including  simulation 
training,  works.  The  Issue  really  Is  one  of 
efficiency  and  effectiveness:  could  more  be 
done  and  could  it  be  done  with  more  efficient 
use  of  resources.  Simulation  plays  an  im¬ 
portant  role  In  pilot  training,  and  this  role 
likely  will  grow.  As  It  grows,  instructor 
selection  and  training  will  be  keys  to  en¬ 
hance  productivity.  Perhaps  it  Is  time  that 
the  principles  of  Instructional  system  de¬ 
velopment  are  applied  to  the  tasks  of  man¬ 
aging  and  conducting  training,  as  well  as  to 
the  tasks  to  be  trained. 
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THEY  CM)  MAKE  OR  BREAK  YOU:  CONSIDERING  THE  INSTRUCTOR 
AS  A  USER  IN  AUTOMATED  TRAINING  SYSTEMS 


Dr.  Robert  Halley 
Log loon,  Inc. 


Military  systems  are  becoming  more 
advanced  and  conplex.  This  results  In  a 
requirement  for  Increased  training  time  by 
users.  Concurrently,  the  growing  manpower 
shortage  is  causing  a  decrease  In  the 
router  of  experienced  people  available  to 
provide  Instruction  and  training  on  any 
new  system.  Thus,  the  military  is  facing 
a  need  to  provide  more  training  using  less 
instructor  resources. 

One  solution  has  been  the  development 
of  automated  training  systems  to  help 
shoulder  the  training  burdens.  These 
training  systems,  more  than  just  simulators 
or  part-task  trainers,  present  design 
problems  which  operational  systems  don't 
have.  Designers  have  to  produce  systems 
which  will  train  the  required  operational 
skills  and  also  provide  training  related 
functions  for  the  students  and  instruc¬ 
tors.  These  include  such  capabilities  as 
on-system  instruction,  performance 
measurement,  preprogrammed  scenarios,  auto¬ 
mated  feedback,  and  training  management 
functions. 

Training  system  design  is  a  fairly 
young  art  and  the  design  artists  are  slowly 
learning  how  to  do  what  they  intend.  In 
the  past,  the  system  designers  have  tended 
to  concentrate  primarily  on  hardware  and 
software  considerations.  More  recently, 
there  has  been  an  increasing  realization 
that  courseware  considerations  (e.g. ,  what 
is  being  trained,  who  is  being  trained, 
etc.)  should  also  be  a  driving  force  in 
training  system  design.  The  latest  reve¬ 
lation  in  system  design  is  "peopleware", 
attention  to  the  people  factors  in  the 
interface  between  man  and  machine. 

Unfortunately,  for  many  designers  the 
term  "users"  only  means  students.  Training 
systems  have  many  other  sets  of  users.  The 
users  include  instructors,  operators,  ad¬ 
ministrators,  and  maintenance  personnel. 
Each  of  these  sets  of  people  will  use  the 
system  and  their  needs  should  be  considered 
in  the  system  desiqn.  Of  this  additional 
set  of  users,  instructors  have  the  most 
influence  on  system  brplementation  and 
success.  As  a  result,  it  is  very  important 
to  design  the  system  to  be  instructor 
friendly. 

There  are  a  large  number  of  factors  to 
consider  in  making  automated  training 


systems  instructor  friendly.  It  is  important 
to  note  that  instructors,  often  for  good 
reason,  tend  to  resist  a  change  from  their 
traditional  approaches  to  ccnputer-based 
training.  First,  they  see  that  they  will 
have  to  learn  a  new  job,  when  they  already 
know  the  old  one  very  well.  Second,  they 
often  perceive  this  change  as  a  demotion 
from  an  important  instructional  position  to 
an  assignment  as  a  ocnputer  system  lackey. 
Third,  there  is  a  cocrnon  popular  fear  about 
having  to  deal  with  things  conputerish. 
Computers  are  typically  regarded  as  being 
mysterious,  expensive,  and  very  breakable. 
This  keeps  people  from  just  stepping  in  and 
using  them.  Fourth,  the  instructors  have 
real  concerns  about  whether  the  new  system 
will  be  a  training  improvement.  They  are 
often  concerned  about  both  instructional 
effectiveness  and  possible  dehumanization 
of  the  training  environment. 

This  instructor  resistance  can  be  ex¬ 
pressed  in  a  variety  of  ways,  anywhere  from 
an  outright  refusal  to  use  the  system  to  a 
subtle  lack  of  faith  in  the  system  which 
gets  transmitted  to  the  student.  All  these 
types  of  resistance  can  lead  to  the  same 
outcome.  No  matter  how  good  a  system  is, 
if  the  instructors  don't  like  and  trust  it, 
they  can  corpletely  destroy  its  effective¬ 
ness.  A  personal  exanple  dates  back  to 
college,  when  a  new  videotape-based  curric¬ 
ulum  was  being  used  to  replace  normal 
lecturing  in  an  oral  conmun icat ions  course. 
The  instructor  wasn’t  convinced  about  the 
new  approach  and  introduced  the  materials 
by  saying  "All  right,  you  guys'll  have  to 
watch  these  dumb  tapes.  Then  we'll  get  to 
the  iirportant  stuff."  Not  many  people 
watched  the  tapes  very  carefully. 

Inplementing  instructor  friendliness  in 
a  system  means  anticipating  the  instructors ' 
needs  and  making  them  as  easy  as  practical 
to  accomplish.  It  is  not  enough  to  put 
together  a  system  which  can  do  many  and  magic 
things  and  not  also  make  it  easy  to  use.  For 
exanple,  irany  systems  have  extensive  error 
checking  to  help  identify  when  problems 
occur,  but  the  system  reports  them  as  a  "TO 
error  106"  or  a  "100  413  218  error"  leaving 
the  user  to  first  go  look  up  in  an  error 
table  precisely  what  happened  and  then  to 
figure  out  what  caused  the  error.  A  more 
user  friendly  approach  would  be  to  have  the 
system  report  something  like  "That  state¬ 
ment  is  incorrectly  formatted.  Please 
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correct  the  statement  and  re-enter  it." 

Practically#  you  need  to  get  the  in¬ 
structors  involved  as  early  in  the  system 
design  process  as  you  can.  Instructors  can 
provide  important  input  in  identifying  which 
tasks  they  will  use  the  system  to  accomplish 
and  that  ways  those  tasks  can  be  done  simply 
and  effectively.  This  information  can  be 
used  in  designing  the  system-instructor 
interface  and  in  preparing  the  instructor 
training  materials.  In  erne  Logical  system 
design#  instructor  pilots  were  utilized  in 
designing  the  system's  information  presen¬ 
tation  approach.  Hie  result  wes  a  very  con¬ 
cise  set  of  CRT  displays  which  have  been 
well  received  as  comprehensive  and  easy  to 
use. 

Instructional  design  is  typically 
accomplished  by  contractors  or  military 
curriculum  designers  working  outside  the 
actual  instructional  context.  They  often 
fail  to  recognize  that  the  instructors  have 
important  things  to  contribute  concerning 
the  curriculum  and  about  the  instructional 
approaches  and  methodologies  that  are 
feasible  and  practical  for  the  student 
population.  It  is  valuable  to  get  in¬ 
structor  input  and  then  make  sure  you  point 
out  their  contribution  in  the  system 
materials.  This  will  add  credibility  to  the 
system  and  give  subsequent  instructors  some 
pride  of  ownership. 

Training  systems  are  often  implemented 
at  user  commands  with  only  a  minimum  of 
accompanying  documentation.  The  documen¬ 
tation  is  sometimes  supplemented  by  a 
minimum  of  user  training  and  then  the  in¬ 
structors  are  left  on  their  own  to  fiqure 
out  the  rest.  The  typical  result  is  that 
the  system  is  exercised  in  only  the 
simplest  modes. 

It  is  important  to  integrate  instructor 
training  into  the  system  for  both  initial 
and  continuing  use.  Ibis  important  tool 
can  affect  the  instructor's  willingness  to 
use  the  system.  It  is  important  to  ensure 
the  instructors'  knowledge  and  attitudes 
about  the  system.  Important  training  topics 
include  (1)  the  continuing  importance  of 
the  instructor's  roles  in  system  success, 

(2)  the  reasons  that  the  system  is  being 
used  to  replace  the  old  methodologies  (3) 
how  the  system  is  used  in  this  instructional 
situation,  and  how  to  use  the  system  to  maxi¬ 
mum  instructional  advantage.  Using  the 
system  to  provide  this  training  can  also 
show  the  instructors  that  the  system  can 
train  effectively. 


This  training  must  provide  the  rationale 
for  the  development  of  the  system  and  present 
the  system's  approach  to  the  Instructional 
task.  The  system  should  also  detail  the 
important  decision-making  roles  that  the 
instructor  will  be  asked  to  fill  and  also 
explain  the  breadth  of  the  system's  capa¬ 
bilities  fo-  instructor  options.  Practice 
should  be  provided  in  utilizing  all  the 
options  and  capabilities. 

The  initial  instructor  training  can  be 
provided  through  a  thorough  instructor 
tutorial  which  covers  all  the  topics  listed 
above  in  an  instructional  package.  This 
should  be  supported  by  a  system  HELP  capac¬ 
ity  which  provides  access  to  individual 
topics  from  within  the  tutorial.  This  would 
allow  a  presentation  of  any  area  in  which 
the  instructor  needs  a  review. 

A  pragmatic  example  of  applying  these 
design  principles  is  the  Log icon  developed 
Instructor  Support  System  (ISS) .  Designed 
specifically  as  a  tool  to  assist  instructors 
with  trainer  associated  tasks,  in  its  initial 
application,  ISS  has  been  attached  to  an 
existing  flight  simulator.  ISS  is  being 
used  to  replace  an  instructor  station  that 
is  so  difficult  to  use  that  the  instructor 
must  spend  most  of  his  energy  interacting 
with  the  instructor  station,  leaving  very 
little  time  or  energy  for  the  task  of  in¬ 
structing. 

The  ISS  provides  many  functions  to 
support  the  instructors'  tasks.  For  example, 
the  instructors  can  choose  from  three 
different  ISS  training  modes:  specialized 
task  training  (SIT) ,  instructor  select 
(ISEL) ,  and  canned.  Each  of  these  training 
modes  allows  the  instructor  to  hand  tailor 
the  students'  simulator  experience  during 
each  exercise.  The  ISS  training  curriculum 
is  comprised  of  a  set  of  training  modules 
such  as  "afterburner  take-off"  or  "Miramar 
TACAN  1  approach"  or  "left  engine  fire." 

Each  module  is  separate  and  includes  soft¬ 
ware  identifying  what  behaviors  are  to  be 
measured,  how  success  is  to  be  graded,  what 
checklists  and  procedures  are  involved,  and 
vhat  marks  the  beginning  and  end  of  the 
task.  In  each  training  mode  these  modules 
are  utilized  differently. 

STT  mode  allows  the  instructor  to 
schedule  one  or  more  training  modules  which 
feature  practice  on  a  specific  skill.  This 
allows  repeated  executions  of  a  task  such 
as  landings  without  having  to  practice  take¬ 
offs  and  other  tasks  associated  with  a 
normal  mission.  Thus,  if  the  instructor 
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felt  the  student  needed  work  on  landings, 
several  landings  in  a  row  could  be  scheduled. 
After  each  landing  the  ccnputer  would 
automatically  set  up  the  next  one  until  the 
session  was  finished. 

ISEL  node  requires  the  instructor  to 
assemble  a  complete  Hchocks-to-chocks" 
mission.  In  either  ISEL  or  SIT  the  in¬ 
structor  can  choose  malfunctions  to  be  in¬ 
serted  into  the  practice. 

The  canned  mode  provides  a  predesigned 
practice  session.  In  this  mode  the 
training  modules  have  already  been  selected 
and  sequenced  and  the  instructor  need  only 
select  the  overall  package.  Since  canned 
mode  practice  sessions  are  generally  de¬ 
signed  to  follow  the  training  syllabus,  the 
canned  mode  helps  to  ease  the  instructors' 
training  burden.  Canned  mode  exercises  are 
also  used  in  check  rides  and  "graduation" 
exercises  to  provide  a  consistent  measure¬ 
ment  environment. 

In  addition  to  assisting  the  instructor 
with  his  task  by  providing  practice  exer¬ 
cise  control,  ISS  provides  other  instructor 
support  features.  These  include  student 
monitoring,  performance  measurement, 
grading,  instructional  records  keeping,  and 
student  performance  debriefing.  The  student 
monitoring  function  allows  the  instructor 
to  watch  a  display  of  what  the  student  is 
doing  while  it  is  being  done.  Inportant 


display  information  including  aircraft  con¬ 
figurations  (e.g. ,  flaps  up,  hook  down) , 
aircraft  parameters  (e.g. ,  speed,  altitude) , 
a  geographic  plot  and  historic  trail  of  air¬ 
craft  position,  and  diagnostic  message  as 
problems  occur  are  clearly  presented  on  two 
display  devices,  rather  than  spread  over  a 
ten-foot  long  instructor  console. 

The  diagnostic  messages  are  a  very  im¬ 
portant  feature  which  appear,  thus  far,  to 
be  unique  to  ISS.  Although  sane  training 
systems  have  diagnostic  messages  which  are 
displayed  at  the  end  of  a  practice  session, 
ISS  provides  a  real-time  diagnostics  display 
at  a  very  detailed  level.  This  allows  the 
instructor  to  knew  immediately  the  precise 
cause  of  a  student  problem.  Figure  1  below 
shews  an  example  of  ISS  diagnostics. 

The  performance  measurement  function 
keeps  track  of  student  behaviors  and  com¬ 
pares  them  to  expected  behaviors.  The 
grading  function  uses  these  performance 
measures  to  provide  a  suggested  grade  for 
the  student  practice  session  on  any  of  a 
number  of  skills.  Since  there  is  often 
some  instructor  resistance  to  having  the 
grading  function  taken  over  by  a  ccnputer, 
ISS  has  been  carefully  designed  so  as  to 
easily  let  the  instructor  review  all  the 
scoring  criteria  and  how  the  student's  grade 
has  been  derived.  Figure  2  on  the  following 
page  shows  a  sanple  of  the  ISS  "grade 
sheet."  The  instructor  can  review  the 
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FIGURE  2.  ISS  GRADE  SHEET  DISPLAY 


session  and  quickly  and  sinply  change  any 
of  the  grades.  The  scoring  criteria  have 
also  been  designed  so  that  they  can  be 
easily  revised  if  the  standards  are  incor¬ 
rect  or  are  changed. 

The  ISS  instructional  records  keeping 
function  stores  the  performance  measure¬ 
ments,  results,  and  grading  for  immediate 
CRT  access  and  as  well  as  for  printing  hard 
copy  records.  The  student  debriefing 
function  is  an  extension  of  the  records 
keeping  capability.  The  student's  last 
entire  practice  session  is  recorded  and 
the  individual  tasks  can  be  played  back  at 
either  normal  or  fast  (4  times  normal) 
speeds  or  can  be  frozen  at  any  point  for 
the  student  and  instructor  to  review.  All 
of  the  aircraft  data  and  student  perfor¬ 
mance  data  are  replicated  during  the  debrief 
playback. 

ISS  playback  is  different  than  that 
found  in  most  similar  systems.  In  never 
training  systems,  for  exarrple,  the  play¬ 
back  mode  recreates  the  cockpit  activities 
such  as  moving  the  stick  and  the  pedals. 

This  information  is  not  very  valuable  in- 
structionally.  In  ISS  the  diagnostics 


messages,  aircraft  parameter  displays,  and 
a  historic  radar  and  connunications  trail 
are  provided  for  review. 

The  ISS  provides  multiple  functions  to 
assist  the  instructor  with  his  job.  Be¬ 
cause  of  careful  use  of  instructor  input 
and  a  continuing  consideration  of  the  in¬ 
structor  as  user  durina  the  design  process, 
the  ISS  provides  a  very  instructor  friendly 
interface.  The  instructions  for  system  use 
are  defined  in  terminoloqv  normally  asso¬ 
ciated.  with  the  aviation  environment .  A 
series  of  decision  menus  are  readily  acces¬ 
sible  through  touch  pad  controls  on  the  lower 
display.  In  contrast  to  many  new  systems 
(e.g. ,  2F112,  2F119)  where  scenario  genera¬ 
tion  is  very  difficult,  these  menus  provide 
the  instructor  with  conplete  exercise 
authoring  and  practice  control  capabilities 
through  a  sirtple  series  of  screen  touch 
interactions. 

The  ISS  encompasses  an  extensive  in¬ 
structor  tutorial  introducing  the  system 
and  the  system's  functions  as  well  as  giving 
the  instructor  an  extensive  series  of  prac¬ 
tice  exercises.  The  tutorial  has  been 
subdivided  into  a  set  of  minitutorials 
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«hlch  are  accessible  at  all  times  through 
a  HELP  function. 

There  are  at  least  two  more  lessons  to 
be  learned  from  the  ISS  experience.  First, 
it  is  not  enough  to  have  the  ISS  features. 
The  features  mist  be  implemented  well  or 
they  can  even  make  the  system  less  usable, 
ftar  exanple,  most  new  systems  provide  in¬ 
formation  on  CRTs  and  provide  system  con¬ 
trol  through  light  pens  or  function  keys. 
The  concept  is  good,  but  the  tendency  is 
to  provide  the  system  access  and  system 
information  without  nudi  thought  to  human 
factors.  Thus,  the  screens  are  crowded  and 
hard  to  read  and  the  menus  hard  to  use. 

Second,  the  ISS  is  not  yet  all  happi¬ 
ness  and  light.  It,  too,  still  has  seme 
problems  in  the  area  of  user  friendliness. 
Design  of  good  practice  sessions  requires 
that  the  instructors  have  done  seme  prior 
planning  and  know  vhat  they  are  going  to 
use  the  session  to  accomplish.  Otherwise 
the  training  session  can  turn  out  somewhat 
haphazard  and  not  meet  its  training  objec¬ 
tives.  These  requirements  for  planning  and 
forethought  can  make  the  ISS  a  little 
scary  to  the  user.  Requiring  this  pre¬ 
planning  may,  in  the  end,  be  beneficial  to 
the  process,  but  it  is  not  entirely  user 
friendly.  An  improvement  on  this  approach 
might  be  to  have  the  system  be  "smart" 
enough  to  help  the  instructor  develop  the 
process  by  having  guidelines  and  rules 
built  in. 

Any  good  training  system  design  will 
attempt  to  design  the  training  system  to 
be  sufficiently  capable  and  flexible  to 
ensure  that  the  machines  serve  the  users 
rather  than  the  other  way  around.  The 
users  should  not  have  to  accomodate  the 
hardware  and  software  by  learning  little 
tricks  and  changing  their  behaviors  to 
meet  the  machine's  needs.  Hovrever,  the 
choice  of  any  approach  will  constrain  the 
ultimate  flexibility  of  the  design  and  its 
ability  to  meet  all  of  the  users'  needs. 
If,  for  exanple,  you  put  the  "ON"  key  on 
the  left,  people  who  are  used  to  having  the 
"CM"  key  on  the  right  or  who  have  never 
had  to  turn  anything  "on"  will  have  to 
learn  to  adapt  to  the  machine.  Money, 
personnel,  and  time  constraints  tend  to 
conspire  to  make  your  design  choices  for 
you,  but  it  is  important  to  remember  to 
build  in  as  much  user  consideration  as  you 
can  and  to  learn  frem  your  mistakes.  ISS, 
in  its  next  incarnations,  is  going  to  pro¬ 
vide  a  mode  where  it  acts  just  like  the 
instructor  console  it  is  replacing.  This 
way  the  instructors  will  have  to  learn 
fewer  new  skills,  but  will  still  have  the 
power  of  the  ISS  at  their  disposal  when 
they  want  it. 


It  is  crucial  to  remember  that  instruc¬ 
tors  are  a  vital  instructional  resource. 
Instructors  can  help  a  training  system  pro¬ 
vide  effective  and  efficient  training  or 
they  can  severely  limit  the  system's  useful¬ 
ness  sinply  by  how  they  respond  to  the 
system.  Instructor  friendliness,  then,  be¬ 
comes  a  major  consideration  in  training 
system  design.  Two  important  initial  steps 
in  building  an  instructor  frigidly  system 
are  getting  to  know  the  instructors'  needs 
and  getting  the  instructors  involved  in  the 
development  process.  A  third  very  important 
step  is  using  the  training  system  to  train 
and  convince  the  instructors  about  the 
system's  usefulness.  A  fourth  important  de¬ 
sign  consideration  is  building  the  system 
to  serve  the  user,  rather  than  vice  versa. 
Lastly,  one  must  remember  the  "user  frigidly" 
includes  all  the  users,  not  just  the  stu¬ 
dents. 
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INTRODUCTION 

Use  of  the  computer  in  its  vari¬ 
ous  manifestations  as  a  medium  for 
instruction  has  contributed  much  to 
our  understanding  of  the  instruction¬ 
al  process.  Impact  of  this  powerful 
medium  on  instructor  roles,  however, 
has  received  little  theoretical  at¬ 
tention.  It  is  the  purpose  of  this 
paper  to  examine  the  impact  of  com¬ 
puter-based  instruction  on  tradi¬ 
tional  instructor  roles  and  functions 
in  order  to  achieve  a  better  under¬ 
standing  of  the  possibilities  it 
creates  for  enhanced  instructor- 
machine  interfacing. 

Use  of  instructional  media  of 
any  kind  tends  to  alter  in  some  ways 
instructors'  roles,  at  least  how  they 
are  performed.  Some  media  may  alter 
the  functions  performed  by  instruc¬ 
tors  to  such  an  extent  that  qualita¬ 
tive  changes  in  roles  results.  It  is 
our  contention  that  such  changes 
occur  wherever  the  computer  is  em¬ 
ployed  as  the  primary  medium  for 
instruction.  Wherever  it  is  used  in 
whatever  mix  with  other  media,  it 
seems  that  the  computer  uniquely 
alters  instructor  functions. 

The  type  of  computer  application 
we  shall  consider  consists  of  a  net¬ 
work  of  terminals  tied  to  a  central 
processing  unit  (CPU) .  In  its  sim¬ 
plest  form  the  system  may  consist  of 
a  single  terminal  devoted  entirely  to 
student  use.  in  more  elaborate  form 
the  system  may  contain  several  termi¬ 
nals,  at  least  one  of  which  would 
serve  as  the  instructor's  console. 

In  even  the  simplest  form  of  the 
system,  there  would  exist  two-way 
communication  between  terminal  and 
CPU.  In  its  more  elaborate  form, 
there  also  may  exist  the  capability 
for  communication  among  terminals 
directly  and  via  the  CPU.  Theoreti¬ 
cally,  the  inputs  to  and  outputs  from 
such  a  network  may  include  virtually 
any  response  a  human  is  capable  of 


making  so  long  as  it  can  be  transdu^ 
ced  at  the  terminal,  and  any  pattern 
of  energy  change  a  human  is  capable 
of  perceiving  if  it  can  be  displayed 
with  fidelity  to  the  human's  senses 
at  the  terminal.  As  futuristic  as 
this  conception  may  appear,  such 
networks  already  form  the  core  of 
multi-terminal  training  systems. 
Indeed,  simplified  versions  may  be 
found  in  your  child's  toy  box.  This 
is  not  to  say  that  limitations  on 
transduction,  processing,  and  dis¬ 
plays  are  not  real,  but  rather  these 
limitations  are  currently  technologi¬ 
cal  as  opposed  to  conceptual.  The 
basic  idea  seems  clear,  but  its  rami¬ 
fications  for  instruction  do  not.  It 
is  to  achieve  a  better  understanding 
of  this  that  we  shall  attempt  to 
explore  the  conception  more  closely, 
especially  as  it  applies  to  the  in¬ 
structor  . 

How  much  of  the  human  instruc¬ 
tor's  traditional  roles  may  be  taken 
over  by  computer-based  instructional 
systems?  What  functions  will  remain 
or  newly  emerge  for  human  instructors 
to  perform?  These  are  questions 
especially  pertinent  to  the  design  of 
instructor  stations. 

For  purposes  of  this  paper  "in¬ 
structor  'station"  is  defined  as  one 
member  of  a  network  of  computer  ter¬ 
minals  driven  by  a  CPU  programmed  for 
interactive  instructional  delivery 
and  two-way  communication  between  the 
instructor's  terminal  and  each  stu¬ 
dent's  terminal.  Clearly,  the  role 
of  an  instructor  operating  such  a 
station  would  differ  significantly 
from  his  role  in  the  traditional 
classroom  even  if  the  instructional 
subject  matter  remained  the  same. 

Before  changes  in  instructor 
roles  and  functions  due  to  use  of  the 
computer  as  an  instructional  medium 
can  be  assessed,  it  seems  necessary 
to  first  arrive  at  some  categoriza¬ 
tion  of  them  as  they  have  been  tradi¬ 
tionally  practiced,  and  to  further 
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examine  the  major  classes  of  factors 
which  determine  their  selection. 

TRADITIONAL  ROLES 

Five  instructor  roles  may  be 
distinguished  as  a  minimum  number  of 
categories  needed  to  describe  the 
major  domains  of  instructor  activity 
within  traditional  settings.  These 


1.  Lecturer 

2.  Leader 

3.  Supervisor 

4.  Tutor 

5.  Aide 

Instructor  roles  are  here  dis¬ 
tinguished  in  terms  of  the  emphasis 
each  gives  to  the  performance  of 
seven  categories  of  instructor  func¬ 
tions. 

TRADITIONAL  FUNCTIONS 

The  major  categories  of  instruc¬ 
tor  functions  in  traditional  instruc¬ 
tional  contexts  may  be  divided  into 
seven  categories.  These  are: 

1.  Information  development 

2.  Information  delivery 

3.  Student  supervision 

4.  Student  guidance 

5.  Student  evaluation 

6.  Data  management 

7.  Course  management 

It  will  be  apparent  in  the  fol¬ 
lowing  definitions  of  traditional 
instructor  functions  that  all  cate¬ 
gories  are  not  mutually  exclusive  of 
one  another.  Even  though  various  of 
these  functions  are  interdependent 
and  overlap  somewhat,  they  neverthe¬ 
less  represent  different  channels 
into  which  instructional  energy  is 
directed.  Function  definitions  are: 

1.  Information  development:  The 
gathering,  synthesizing,  and  organi¬ 
zing  of  subject  matter  (facts,  con¬ 
cepts,  principles,  procedures,  etc.) 
pertinent  to  course  objectives  into 
formats  appropriate  for  presentation 
in  particular  instructional  situa¬ 
tions.  In  many  industrial  and  armed 
services  situations,  more  formal 
approaches  to  instructional  systems 
development  (ISD)  would  be  taken, 
beginning  with  task  analysis. 

2.  Information  delivery:  The 
presentation  of  subject  matter  neces¬ 
sary  to  achieve  course  objectives  by 


whatever  means  that  are  available, 
effective,  and  consistent  with  the 
instructor's  role  and  situation. 

3.  Student  supervision:  The 
direction  of  student  learning  activi¬ 
ties  toward  timely  realization  of 
course  objectives  through  mandate, 
directive  advice,  personal  example, 
assignment  of  study  materials,  stipu¬ 
lation  of  practice  techniques,  per¬ 
formance  evaluation,  praise,  criti¬ 
cism,  etc. 

4.  Student  guidance:  The  assis¬ 
tance  of  students  in  discovering, 
orienting  toward,  and  developing 
feasible  approaches  to  realization  of 
future  goals  through  counseling, 
interpretative  evaluation  of  apti¬ 
tudes,  providing  information  perti¬ 
nent  to  formulation  and  realization 
of  long-term  objectives,  etc. 

5.  Student  evaluation:  Assess¬ 
ments  of  relative  performance,  sub¬ 
ject  matter  and/or  skills  mastery, 
and  any  other  dimension  of  behavior 
correlated  with  success  in  an  area  of 
activity  (attitudes,  motivation, 
emotional  stability,  etc.)  on  the 
basis  of  test  scores,  proficiency 
scores,  subjective  ratings,  physio¬ 
logical  indices,  etc. 

6.  Data  management:  The  record¬ 
ing,  categorizing  and  filing  of  indi¬ 
vidual  performance  data;  calculations 
of  norms,  trends,  statistics,  etc.; 
documentation  of  instruction-related 
student  activities  (attendance, 
promptness,  awards,  demerits,  etc.); 
collation  of  summary  data  for,  and 
preparation  of,  student  progress 
reports;  etc. 

7.  Course  management:  Control¬ 
ling  the  conduct  of  a  course  of  in¬ 
struction  including  all  decisions 
regarding  subject  matter,  examples, 
demonstrations,  etc.,  to  be  presen¬ 
ted,  the  study  materials  and  practice 
exercises  to  be  required,  the  in¬ 
structional  media  to  be  used,  the 
tests  and  other  evaluative  instru¬ 
ments  to  be  used,  the  sequencing  and 
time  allotments  for  instructional 
segments,  the  updating  and  ’-evisions 
of  course  objectives,  syllr.  ii,  and 
instructional  materials,  etc.,  the 
allocation  of  instructional  resources 
(funds,  supplies,  etc.)  and  facili¬ 
ties,  etc. 


ROLES  AND  FUNCTIONS 

Roles  are  not  sharply  delineated 
in  terms  of  the  traditional  instruc¬ 
tor  functions,  but  rather  in  terms  of 
the  ways  the  functions  are  performed 
and  the  level  of  responsibility  each 
role  assumes  for  the  execution  of  a 
given  function.  Not  all  of  the  tra¬ 
ditional  functions  are  present  in 
every  instructor  role,  e.g.,  aides 
are  usually  not  responsible  for  in¬ 
formation  development  and  lecturers 
are  minimally,  if  at  all,  responsible 
for  student  guidance.  Even  where  two 
roles  are  responsible  for  the  same 
type  of  function,  they  would  rarely 
carry  them  out  in  the  same  manner, 
e.g.,  the  delivery  of  instructional 
information  by  lecturers  and  super¬ 
visors  is  entirely  different. 

Both  the  level  of  responsibility 
for,  and  the  manner  of  performing 
each  of  the  traditional  instructor 
functions  may  be  viewed  as  outcomes 
of  a  deterministic  process. 

DETERMINANTS  OF  FUNCTIONS 

There  appear  to  be  four  immedi¬ 
ate  determinants  of  instructor  func¬ 
tions.  These  are: 

1.  Instructional  objectives 

2.  Instructional  media 

3.  Delivery  situations 

4.  instructor/student  ratios 

Usually,  the  above  conditions 
are  established  by  a  more  remote 
group  of  determinants,  including: 

1.  Instructional  goals 

2.  Entry-level  requirements 

3.  Through-put  requirements 

4.  Exit-proficiency  requirements 

5.  Resources  and  facilities 

6.  Time/Cost  constraints 

While  the  above  by  no  means 
exhausts  the  realm  of  potential  de¬ 
terminants,  immediate  or  remote,  they 
seem  to  be  the  ones  that  are  most 
influential  in  most  instructional 
settings.  Consider  the  following 
examples: 

Example  1.  Assume  that  the 
instructional  goal  of  a  university's 
department  of  philosophy  is  to  offer 
instruction  in  philosophy  of  science 
at  a  level  which  could  be  taken  by 
graduate  students  in  all  areas  of 
science.  Entry-level  requirements  in 
this  case  would  be  quite  general, 


limited  to  students  with  graduate 
status,  but  stipulating  no  specific 
prerequisites.  University  and  de¬ 
partmental  policies  would  determine 
the  through-put  requirements;  a  mini¬ 
mum  and  maximum  acceptable  number  of 
students  per  academic  term.  Exit- 
proficiency  requirements  could  be  set 
by  a  committee  of  experts  consistent 
with  the  instructional  goal,  but 
usually  this  would  be  done  by  the 
instructor ( s) .  Resources  and  facili¬ 
ties  would  include  instructional 
manpower,  library  holdings,  media, 
rooms,  etc. 

In  order  to  satisfy  the  instruc¬ 
tional  goal,  a  high  level  of  exper¬ 
tise  would  be  required  of  the  in¬ 
structor,  usually,  a  Ph.D.  with  a 
specialty  in  the  philosophy  of  sci¬ 
ence.  Few  universities  would  have 
more  than  one  such  individual  avail¬ 
able  and  it  is  unlikely  that  the 
university  could  afford  to  bring  in 
additional  instructors.  The  bottom 
line  would  be  that  this  course  of 
instruction  would  be  offered  by  one 
instructor  and  that  the  delivery 
situation  would  be  a  classroom  with 
media  support  restricted  to  that  on 
hand  (slide  projector,  blackboard, 
textbooks,  mimeographed  handouts, 
etc.).  This  would  not  be  a  serious 
handicap  since,  in  this  case,  the 
instructional  objectives  would  be 
conceptual  rather  than  skilled.  At 
this  point  all  major  aspects  of  the 
instructor's  functions  have  been 
determined. 

The  cost  constraint  has  limited 
the  number  of  instructors  to  one  and 
the  through-put  requirements,  to¬ 
gether  with  the  available  delivery 
situation,  has  set  the  acceptable 
number  of  students  at,  say,  between 
15  and  30.  The  resulting  small  in¬ 
structor/student  ratio  combined  with 
a  single  instructor  delivering  ab¬ 
stract  conceptual  material  in  a  class 
room  situation  with  only  rudimentary 
media  support  would  mean  the  follow¬ 
ing  : 

a)  The  instructor's  responsi¬ 
bility  for  information  development 
would  be  nearly  total  and  probably  it 
would  have  to  be  carried  out  person¬ 
ally,  the  result  depending  heavily  on 
the  level  of  specialized  expertise  of 
which  the  instructor  is  capable; 

b)  The  instructor  would  bear 
total  responsibility  for  delivery  of 
subject  matter  information  in  the 
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classroom,  and,  due  to  the  limited 
support  media  at  his  disposal  and  the 
number  of  students  he  must  reach,  the 
only  mode  available  for  information 
delivery  would  be  that  of  the  lec¬ 
ture; 

c)  The  very  conditions  that  dic¬ 
tate  the  lecture  form  of  information 
delivery  would  pre-empt  one-to-one 
student  interaction  (unless  it  occur¬ 
red  outside  of  class)  and,  therefore, 
would  render  negligible  any  presump¬ 
tion  of  responsibility  by  the  in¬ 
structor  for  the  functions  of  student 
supervision  or  student  guidance; 

d)  Unless  the  instructor  were 
assigned  an  assistant  to  evaluate 
assignments,  grade  tests,  and  keep 
track  of  attendance,  etc.,  the  in¬ 
structor's  responsibility  for  the 
functions  of  student  evaluation  and 
data  management  would  be  total.  In 
the  case  that  an  assistant  were  avail¬ 
able  for  these  duties,  the  instructor's 
responsibilities  could  be  reduced  to 

a  minimum  depending  upon  the  competence 
of  the  assistant; 

e)  The  instructor  would  assume 
complete  responsibility  for  develop¬ 
ing  the  course  syllabus,  determining 
when  and  how  much  time  will  be  devo¬ 
ted  to  each  instructional  section  of 
the  course,  the  nature  and  number  of 
assignments  and  tests  to  be  given, 
selection  of  textbooks  and  other 
reading  materials,  and  all  other 
decisions  affecting  the  conduct  of 
the  course. 

It  is  apparent  that  the  job 
description  which  emerges  from  the 
above  enumeration  of  functions  could 
only  be  that  of  a  lecturer.  The 
roles  of  leader  and  tutor  would  en¬ 
tail  similar  levels  of  responsibility 
for  information  development  and  de¬ 
livery,  as  well  as  for  student  evalu¬ 
ation  and  the  management  functions, 
but  the  levels  of  responsibility 
assumed  by  leaders  and  tutors  for 
student  supervision  and  guidance 
would  be  considerably  greater  than 
that  afforded  by  the  determinants. 

In  the  example  under  considera¬ 
tion,  the  instructor  could  be  a  (dis¬ 
cussion)  leader  if  the  instructor/ 
student  ratio  were  reduced  thus  enab¬ 
ling  a  more  informal  mode  of  informa¬ 
tional  delivery.  In  order  for  the 
instructor  to  serve  as  a  tutor  (per¬ 
mitting  a  highly  interactive,  one- 
on-one  mode  of  information  delivery) 


the  university's  through-put  require¬ 
ments,  resources  and  restraint  allo¬ 
cations,  perhaps  even  its  instruc¬ 
tional  goals,  would  have  been  quite 
different.  Certainly,  requirements 
for  student  supervision  and  guidance 
in  association  with  this  course  of 
instruction  would  have  been  far  more 
compelling  than  they  were.  The  point 
is  that,  given  the  assumptions  in 
this  example,  the  only  role  that  is 
feasible  within  this  traditional 
instructional  context  is  that  of 
lecturer . 


It  should  be  noted  here  that  a 
distinction  can  be  made  between  two 
kinds  of  instructional  leaders.  The 
first  kind  is  that  mentioned  in  the 
example  above,  i.e.,  the  discussion 
leader.  The  second  kind  is  what  we 
shall  call  the  field  (or  team)  lead¬ 
er.  Both  provide  instruction  for 
relatively  small  groups  of  students 
and  the  levels  of  responsibility 
assumed  for  each  of  the  seven  in¬ 
structor  functions  is  approximately 
the  same.  (Incidentally,  the  ranges 
of  levels  of  responsibility  assumed 
by  leaders  for  the  various  functions 
appears  to  be  more  variable  than  in 
the  case  of  any  other  instructor 
role.)  The  chief  differences  between 
field  and  discussion  leaders  are  that 
the  latter  serves  within  classroom¬ 
like  situations  and  the  former  relies 
heavily  on  personal  example,  usually 
with  some  authority  beyond  that  of 
instruction . 

Example  2.  Assume  that  the 
instructional  goal  of  a  major  airline 
is  to  train  technicians  in  the  main¬ 
tenance  of  a  particular  kind  of  jet 
engine.  The  entry-level  requirements 
for  students  to  be  admitted  to  this 
program  include  previous  training  in 
basic  electricity  theory,  circuits 
design  and  construction,  blueprint 
reading,  transformers  and  motors, 
electro  mechanical  devices,  micro¬ 
processor  controls  and  operations, 
etc.  The  through-put  requirements 
call  for  six  students  to  be  trained 
in  an  11-week  apprent iceship-type 
training  program.  Exit-proficiency 
requirements  stipulate  that  each 
student  must  be  capable  of  performing 
all  normal  maintenance  functions, 
trouble  shooting  electrical  circuits, 
carrying  out  performance  tests  and 
measurements,  making  minor  repairs, 
etc.  Physical  resources  and  facili¬ 
ties  are  available  at  a  centralized 
company  maintenance  school.  The 


information  delivery  situation  will 
be  a  skills  laboratory  with  such 
media  as  engine  mock-ups,  engine 
circuit  mock-ups  with  programmable 
malfunctions,  testing  instrumenta¬ 
tion,  individual  student  study  car- 
rells  equipped  with  audio-visual 
devices,  student  notebooks  containing 
job-aides,  etc. 

Although  the  cost  for  skills 
laboratory  and  media  are  consider¬ 
able,  they  are  made  feasible  by  con¬ 
centrating  the  training  into  one 
well-equipped  centralized  facility. 
This  also  reduces  the  number  of  in¬ 
structors  needed  to  one.  The  in¬ 
structor's  qualifications  include 
several  years  experience  in  aircraft 
maintenance  plus  successful  comple¬ 
tion  of  an  extensive  instructor 
training  program  conducted  by  the 
manufacturer  of  the  engine  in  ques¬ 
tion.  It  may  be  assumed  that  the 
company's  maintenance  training  pro¬ 
gram  is  a  somewhat  streamlined  ver¬ 
sion  of  the  manufacturer's  instructor 
training  program.  The  instructional 
objectives  would  consist  almost  en¬ 
tirely  of  concept  applications  and 
hands-on  skills. 

The  instructor's  responsibilities 
for  information  development  in  this 
case  would  be  negligible,  but  he 
would  be  almost  totally  responsible 
for  information  delivery,  student 
supervision,  student  guidance,  stu¬ 
dent  evaluation,  and  data  management. 
The  instructor's  level  of  responsi¬ 
bility  for  course  management  would  be 
moderate  since  all  decisions  regard¬ 
ing  sequencing  of  instructional  units, 
testing,  evaluation,  etc.  would  have 
been  made  at  the  time  of  training 
program  development.  Given  this 
array  of  function  responsibilities 
and  their  determining  conditions,  the 
role  of  this  instructor  could  only  be 
that  of  a  supervisor. 

Although  this  instructor  may, 
from  time  to  time,  carry  on  group 
discussions  with  his  students,  his 
role  could  not  be  that  of  a  discus¬ 
sion  leader  because  the  instructional 
objectives  dictate  that  he  personally 
supervise  hands-on  exercises  in  a 
skills  laboratory.  Neither  could 
this  instructor  be  regarded  as  a 
tutor  because  he  has  no  responsi¬ 
bility  for  information  development, 
is  minimally  concerned  with  general 
theoretical  knowledge,  and  he  is 
required  to  demonstrate  and  oversee 
the  acquisition  of  skills  by  more 


than  one  student.  Superficially,  the 
supervisor's  role  appears  most  like 
that  of  the  aide  in  that  the  latter 
might  well  work  with  the  same  number 
of  students  under  much  the  same  con¬ 
ditions.  However,  in  this  example, 
an  aide  would  have  little  responsi¬ 
bility  for  information  delivery  and 
moderate  to  low  responsibility  for 
student  supervision,  guidance,  and 
evaluation.  In  this  situation  an 
aide's  responsibility  for  data  manage¬ 
ment  could  range  between  high  and 
low,  and  relatively  little  responsi¬ 
bility  would  exist  for  course  manage¬ 
ment.  Thus  the  configuration  of 
responsibilities  assumed  for  in¬ 
structor  functions  in  this  example 
clearly  limits  the  choice  of  roles  to 
that  of  a  supervisor. 

The  relationships  between  deter¬ 
minants,  functions,  and  instructor 
roles  are  both  interesting  and  com¬ 
plex,  involving  considerations  other 
than  just  those  mentioned  in  the 
above  examples.  One  such  considera¬ 
tion  is  the  level  and  type  of  exper¬ 
tise  that  is  generally  required  for 
each  role.  Since  expertise  is  usu¬ 
ally  inversely  correlated  with  supply 
in  the  market  place,  within  limits 
expertise  can  be  translated  into 
dollar  cost.  However,  when  instruc¬ 
tor  costs  are  added  to  the  total 
costs  of  instructional  delivery  and 
the  cost  per  student  is  calculated, 
the  cost  of  instructor  expertise  in 
some  roles  may  be  markedly  dimini¬ 
shed.  While  a  highly  qualified  lec¬ 
turer  may  be  expensive  relative  to  a 
supervisor,  the  cost  per  student  for 
the  lecture-type  instruction  usually 
would  be  less  due  to  the  larger  num¬ 
ber  of  students  and  the  absence  of 
any  need  for  expensive  media.  Even 
so,  the  matter  of  instructor  exper¬ 
tise  seems  to  be  more  a  problem  of 
supply  than  cost.  In  fact  it  appears 
that,  the  higher  the  level  of  re¬ 
quired  expertise  is,  the  shorter  the 
supply  of  qualified  instructors. 

Instructor  availability  is  one 
factor  that  is  directly  impacted  by 
modern  instructional  technology, 
especially  computer-based  instruc¬ 
tion.  If  the  supply  of  instructors 
in  any  given  field  is  reduced  or  even 
limited  by  the  time  it  takes  to 
train,  or  educate,  those  instructors, 
then  any  approach  to  instruction 
which  can  replace  at  least  some  in¬ 
structor  functions  with  automated 
functions  should  decrease  the  time 
required  to  produce  the  needed 
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supply.  This  is  not  to  say  that  the 
number  of  months  or  years  it  takes  to 
train  effective  instructors  is  the 
only  factor  affecting  their  supply  at 
any  point  in  time.  Monetary  and 
career  incentives,  job  satisfaction, 
aptitude  and  previous  education  are 
among  the  major  factors  which  control 
the  number  of  individuals  who  are 
motivated  and  competent  to  receive 
instructor  training  in  the  first 
place.  In  this  area  also,  compu¬ 
ter-based  systems  should  improve 
supply  through  incentives  and  job 
satisfaction.  Time  savings  in  train¬ 
ing,  and  increased  motivation,  of 
instructors  are  two  factors  which 
should  prove  especially  important  to 
instructional  systems  that  experience 
high  rates  of  instructor  turnover, 
such  as  those  in  the  armed  services. 

It  is  not  the  purpose  of  the 
present  paper  to  examine  the  ways  in 
which  computer-based  instructional 
systems  provide  motivational  incen¬ 
tives  and  increased  job  satisfaction 
for  the  instructors  that  operate 
them,  but  some  of  these  ways  will 
become  apparent  later  on  in  this 
paper.  Suffice  it  to  say  that  the 
possibilities  for  instructor  motiva¬ 
tion  inherent  in  the  network-type 
computer-based  system  is  an  exciting 
and  important  frontier  for  future 
development.  It  is  the  other  avenue 
of  impact  that  the  computer-based 
system  has  on  instructor  supply  (in¬ 
deed,  instructor  effectiveness  also) 
with  which  we  are  concerned  here, 

i.e.,  the  alteration  of  instructor 
functions. 

Which  of  the  traditional  func¬ 
tions  will  be  altered?  Each  of  the 
functions  will  be  altered  in  some 
ways,  and  several  may  be  virtually 
eliminated.  However,  one  further 
aspect  of  traditional  instruction 
needs  some  mention  before  proceeding 
to  an  examination  of  automated  in¬ 
structor  functions.  That  is,  the 
relationship  between  traditional 
instructional  delivery  situations  and 
instructor  roles. 

TRADITIONAL  SITUATIONS 

The  physical  situations  in  which 
instruction  is  traditionally  de¬ 
livered  constrains  instructor  func¬ 
tions  and  thus  narrows  the  choice  of 
roles  appropriate  to  be  performed  in 
them.  Five  major  classes  of  instruc¬ 
tional  delivery  situations  may  be 
distinguished.  These  are: 


1.  Classroom 

2.  Skills  laboratory 

3.  Individualized  study-station 

4.  Simulated  operational  en- 
v i ronment 

5.  Actual  operational  environ¬ 
ment 

It  is  probably  unnecessary  to 
enumerate  the  distinguishing  charac¬ 
teristics  of  each  of  these  situations 
since  they  are  generally  familiar. 
Certainly,  this  classification  scheme 
could  be  expanded  if  finer  distinc¬ 
tions  were  made.  The  scheme  offered 
here  is  the  minimal  number  of  cate¬ 
gories  of  situations  in  which  in¬ 
struction  is  traditionally  delivered 
and  which  permits  the  distinctions  we 
wish  to  make. 

The  degree  and  kind  of  interac¬ 
tions  between  students  and  instruc¬ 
tors  differ  in  these  situations, 
i.e.,  the  emphasis  on  functions  and 
the  way  they  are  carried  out  change 
as  do  instructor  roles.  instructors 
simply  do  not  lecture  in  situations 
3,  4,  and  5.  Only  rarely  would  a 
lecture  occur  in  situation  2.  It 
seems  that  the  lecture  form  of  in¬ 
structional  delivery  is  largely  con¬ 
fined  to  situation  1.  Thus  it  goes 
without  saying  that  the  lecturer  is 
at  home  only  in  the  classroom.  Like¬ 
wise,  the  discussion  leader  also 
finds  his  primary  place  there.  The 
role  of  field  leader,  on  the  other 
hand,  seems  to  occur  with  greatest 
frequency  in  actual  operational  en¬ 
vironments,  but  this  role  would  not 
be  uncommon  in  simulated  environ¬ 
ments.  Of  course,  the  supervisor  is 
most  at  home  in  the  skills  laboratory 
but  he  may  be  found  in  situations  4 
and  5.  Tutors,  which  appear  to  be 
decreasing  in  their  frequency  of 
appearance,  probably  due  to  their 
high  cost  per  student,  carry  out 
their  functions  only  in  individual- 
ized-stations.  By  contrast,  aides 
appear  to  be  the  most  ubiquitous  of 
instructors,  occurring  in  all  situ¬ 
ations  except  the  classroom. 

The  changes  in  roles  with  situ¬ 
ations  reflect  changes  in  other  tra¬ 
ditional  determinants  as  well.  In¬ 
structional  media,  objectives,  de¬ 
livery  techniques,  and  instructor/ 
student  ratios  also  change  with  situ¬ 
ations.  The  covariance  of  these 
immediate  determinants  of  instructor 
functions  is  attributable  to  the 
combined  influences  of  the  conditions 
referred  to  in  the  last  section  as 


remote  determinants.  The  flow  of 
influence  between  remote  and  immediate 
determinants,  however,  is  not  always 
unidirectional,  i.e.,  from  remote  to 
immediate  to  functions  and  ultimately 
to  roles.  Rather,  the  flow  of  influ¬ 
ence  occasionally  is  two-way,  in  both 
the  vertical  and  horizontal  directions. 
For  example,  if  the  situation  we  have 
is  a  one-room  schoolhouse,  just  money 
enough  for  one  instructor  who  must 
meet  state  qualification  standards,  a 
blackboard,  a  couple  boxes  of  chalk, 
a  closet  full  of  dog-eared  books,  and 
30  or  so  right-handed  chairs,  then 
the  instructional  goals  and  other 
remote  determinants  have  to  be  modi¬ 
fied  to  suit  the  more  immediate  ones, 
if  they  are  considered  at  all.  It  is 
inescapable  that,  in  this  situation, 
the  instructor  is  going  to  be  a  lec¬ 
turer  regardless  of  what  behaviors 
are  to  be  trained  (taught) . 

If  there  is  any  truth  to  the 
assertion  that  the  behaviors  which 
can  be  trained  in  any  situation  de¬ 
pend  on  the  behaviors  which  can  be 
brought  to  occur  there,  then  it  must 
also  be  true  that  instructional  de¬ 
livery  situations  substantially  in¬ 
fluence  training  effectiveness  since 
they  certainly  limit  (if  not  induce) 
much  of  the  behavior  that  does  occur 
in  them.  Unfortunately,  too  much  of 
what  appears  to  occur  in  the  school¬ 
room  today,  seems  to  be  learned,  the 
surprise  being  that  anything  else 
(e.g.,  academics)  could  be.  An  an¬ 
swer  to  this  problem  that  one  so 
often  hears  calls  for  "more  disci¬ 
pline.”  It  would  seem  that  this 
sentiment  is,  if  nothing  else,  at 
least  in  the  right  direction  for  it 
implies  control  of  behavior.  We 
suggest  that  it  is  the  interactions 
with  subject  matter  that  needs  most 
to  be  controlled.  It  is  just  this 
kind  of  control  that  automated  sys¬ 
tems  can  be  effective  in  providing. 

AUTOMATED  FUNCTIONS 

In  this  section  we  summarize 
what,  in  general,  a  computer-based 
instructional  system  can  provide  in 
the  place  of  each  of  the  traditional 
instructional  functions.  The  lists 
of  automated  functions  presented  here 
are  state-of-the-art.  As  exciting  as 
current  capabilities  are  for  instruc¬ 
tional  application,  it  should  be  kept 
in  mind  that  considerable  progress 
remains  to  be  achieved  in  all  areas 
of  this  man-machine  interface  (re¬ 
sponse  transduction,  intelligent 


programs,  information  displays). 
Although  some  of  the  automated  func¬ 
tions  listed  below  can  be  performed 
by  relatively  small  systems,  the  type 
system  we  are  considering  consists  of 
a  network  of  sophisticated  terminals 
(both  student  and  instructor)  driven 
by  an  imaginatively  programmed  large- 
capacity  CPU. 


1 .  Development  of  instructional 
information:  (a)  serves  as  a  guide  to 
instructional  development  by  means  of 
programs  (menus  with  prompts  and 


messages)  based  on  algorithms  for 
each  successive  stage  of  course  de¬ 
sign;  (b)  facilitates  writing,  edit¬ 
ing,  and  drawing  of  instructional 
materials  through  programs  for  word 
processing  and  graphics  production; 

(c)  permits  convenient  filing,  cross- 
referencing,  and  combining  of  in¬ 
structional  information  through  pro¬ 
grams  for  information  management. 


2 .  Delivery  of  instructional 
information:  (a)  presents  course 
subject  matter  ranging  from  abstract 
concepts  to  factual  itemizations  in 
self-paced,  mastery-based  formats 
displayed  in  written,  spoken,  and/or 
graphic  forms;  (b)  delineates  rele¬ 
vance  and  applicability  of  subject 
matter  by  presenting  contextural 
information  and  sample  problem  solu¬ 
tions,  problem-solving  exercises, 
etc.;  (c)  provides  instructions  for 
proceeding  through  programmed  les¬ 
sons,  performing  skills,  correcting 
mistakes,  or  obtaining  remedial  in¬ 
formation  for  review;  (d)  produces  or 
controls  simulated  representations  of 
operational  devices,  field  or  job 
situations,  abstract  processes,  per¬ 
formance  procedures,  job  aides,  etc., 
enabling  demonstrations,  practice, 
rehearsal,  etc.,  of  concepts,  rules, 
skills,  procedures,  attitudes,  roles, 
team  exercises,  "what  if"  explora¬ 
tions,  etc.;  (e)  delivers  individual 
and/or  group  response-contingent 
feedback  designed  to  aide  self-diag¬ 
nosis  of  learning  progress;  informa¬ 
tion  may  consist  of  prompts,  ques¬ 
tions,  encouraging  messages,  scores, 
etc.,  in  a  variety  of  display  formats 
that  may  include  special  auditory  or 
visual  effects;  (f)  provides  summary 
feedback  at  the  end  of  each  instruc¬ 
tional  unit  or  major  exercise  con¬ 
sisting  of  scores,  outcome  state¬ 
ments,  evaluations  of  relative  per¬ 
formance,  recommendations  for  im¬ 
provement,  overall  course  performance 
profiles,  course  grades,  etc. 
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3.  Student  supervision;  (a) 
guides  students  along  instructional 
paths  adjusted  for  level  of  achieve¬ 
ment  and  rate  of  progress;  (b)  opti¬ 
mizes  individualized  instructional 
paths  through  frequent  performance 
checks,  variable  path  branching,  and 
review  sequences;  (c)  provides  super¬ 
visory  instructions  and  feedback 
(written,  spoken,  and/or  visual; 
graphical  or  simulated  visual  demon¬ 
strations  of  "do  this")  contingent 
upon  individual  or  group  actions  with 
possible  instructor  intervention; 

4.  Student  guidance:  (a)  pro¬ 
vides  recommendations  for  future 
courses  of  action  (remedial  study, 
more  in-depth  study,  information 
sources,  job  possibilities,  etc.) 
based  upon  performance  profiles;  (b) 
response  to  student  questions  about 
their  performance  profiles  with  in¬ 
terpretive  answers  and  comparative 
data;  (c)  provides  job  descriptions 
(requirements,  work  conditions,  sal¬ 
ary  data,  etc.)  in  areas  related  to 
course  of  instruction. 

5 .  Student  evaluation :  ( a ) 

determines  correctness  of  choices, 
problem  solutions,  or  actions  on  an 
item-by-item  basis;  (b)  tests  per¬ 
formance  proficiency  relative  to 
instructional  objectives,  group 
norms,  and  standards;  (c)  diagnoses 
learning  progress,  detects  learning 
difficulties  early,  prescribes  reme¬ 
dial  work,  and  adjusts  difficulty 
levels/rates  to  match  students'  a- 
bilities;  (d)  provides  overall  per¬ 
formance  profiles,  course  grades, 
percentile  ranks,  etc. 

6.  Data  management:  (a)  accepts 
as  inputs  any  properly  computer- 
interfaced  responses  by  students  and 
instructors;  (b)  automatically 
records  in  central  memory  input  data 
from  all  system  terminals,  pools 
group  data,  forms  generic  data  bases 
and  carries  out  statistical  or  other 
processing  while  preserving  indi¬ 
vidual  student  records;  (c)  displays 
selected  data  files,  analysis  results, 
or  interpretative  messages  auto¬ 
matically,  or  on  command  to  designated 
terminal  in  written,  spoken,  and/or 
graphical  formats. 

7.  Course  management:  (a)  de¬ 
scribes  syl  labus-control  led  sequen¬ 
cing  of,  and  time  allocations  for, 
successive  instructional  units  and 
the  timely  execution  of  tests,  evalu¬ 
ations,  and  student  feedback;  (b) 


tracks  individual  and  group  progress 
relative  to  established  course  mile¬ 
stones,  performance  standards,  and 
through-put  rates;  (c)  carries  out 
course  evaluations  and  recommenda¬ 
tions  for  revision  based  on  data 
analysis  and  success  in  attaining 
instruct ional  objectives;  (d)  enables 
monitoring  of  individual  or  group 
performance  by  instructor  who  may 
selectively  intervene  or  interact 
with  individuals  or  group;  (e)  pro¬ 
vides  prompts  and  messages  to  in¬ 
structors  to  insure  t imely  occurrence 
of  non-computer  instructor  functions. 

The  above  lists  of  automated 
functions  include  no  mention  of  how 
they  are,  or  might  be,  effected. 

Even  though  this  question  is  cer¬ 
tainly  beyond  the  scope  of  this  paper 
there  are  at  least  three  good  reasons 
why  it  is  worthwhile  to  consider 
automated  functions  independently  of 
the  technological  means  by  which  they 
can,  or  might  be,  achieved;  (a)  appli 
cations  of  technology  must  be  justi¬ 
fied  in  terms  of  the  functions  it  can 
perform,  and  those  functions  must 
stand  on  then  own  merits;  (b)  the 
same  functions  may  be  accomplished  in 
different  ways  depending  on  the  re¬ 
quirements  of  particular  applica¬ 
tions,  and  (c)  new  applications  of 
the  same  functions  and  new  techniques 
for  effecting  them  may  be  developed. 
It  is  also  the  case  that  new  tech¬ 
nologies  result  in  the  emergence  of 
previously  unanticipated  functions 
and  applications.  As  we  shall  show, 
applications  of  computer  technology 
in  instruction  modifies  instructor 
roles  not  only  by  assuming  and  im¬ 
proving  upon  traditional  instructor 
functions,  but  also  by  laying  the 
foundation  for  emergence  of  new  in¬ 
structor  functions.  But  first,  let 
us  compare  automated  and  traditional 
instructor  functions. 

FUNCTION  DIFFERENCES 

Perhaps  the  appropriate  question 
at  this  point  would  be,  what  instruc¬ 
tor  functions  can,  and  must,  humans 
perform  that  computerized  machines 
cannot?  In  answer,  we  must  admit  to 
ignorance.  It  is  the  word  "must"  in 
the  question  that  renders  its  answer 
obscure,  at  least  to  us.  Even  if  we 
could  enumerate  evety  possible  human 
activity  that  might  be  construed  to 
be  an  instructor  function,  we  could 
not  say  on  the  basis  or  scientific 
evidence  which  of  them  wore  essential 
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to  the  learning  process.  Perhaps 
none  are. 

It  would  seem  safe  to  conclude 
from  the  last  half-century's  research 
on  the  learning  process  that  the 
minimum  necessary  conditions  for 
learning  are  (a)  the  presentation  of 
energy-borne  information  to  the  senses 
of  a  living  organism,  and  (b)  the 
consequent  occurrence  of  some  sensory 
effect  within  the  organism  that  may 
be  overtly  manifested  in  its  behav¬ 
ior.  While  these  two  conditions  are 
necessary,  even  if  they  are  met  the 
occurrence  of  learning  is  not  as¬ 
sured.  Information  enhancement, 
repetition  of  its  presentation,  re¬ 
sponse-feedback,  etc.,  also  may  be 
required  in  certain  circumstances  to 
increase  the  probability  that  learn¬ 
ing  will  occur,  that  it  will  occur 
rapidly,  and  that  it  will  endure. 
Whether  or  not  such  conditions  are 
essential  to  the  learning  process  in 
some  fundamental  theoretical  sense 
(if,  indeed,  there  is  just  one  learn¬ 
ing  process) ,  it  seems  they  must  be 
considered  practically  indispensible 
in  circumstances  appropriate  for  the 
accomplishment  of  instruction.  Since 
instructor  functions  traditionally 
have  been  the  means  through  which 
production  and  manipulation  of  these 
conditions  for  learning,  has  been 
effected,  it  seems  reasonable  to 
assume  that  any  instructor  function 
which  a  computerized  machine  cannot 
be  programmed  to  simulate  effectively 
should  be  considered  a  "must"  for  the 
human  instructor.  Some  insight  into 
what  these  "must"  functions  should  be 
may  be  gained  by  examining  the  dif¬ 
ferences  between  instructor  and  auto¬ 
mated  functions.  Function  differ¬ 
ences  are  summarized  below  using  the 
same  numbering  of  categories  as  be¬ 
fore. 

1.  Computer-based  systems  can 
facilitate  the  development  of  in¬ 
structional  information,  but  they 
cannot  recognize  instructional  needs, 
originate  program  goals,  or  conceive 
of  the  means  by  which  to  attain  them. 
Humans  must  assume  virtually  all 
responsibility  for  gathering  and 
synthesizing  information,  evaluating 
what  would  be  important  and  interest¬ 
ing,  creating  stimulating  conceptu¬ 
alizations,  examples,  etc.,  of  the 
information,  and  finally,  originating 
innovative  computer  programs  to 
achieve  effective  information  delivery 
to,  and  interaction  with,  students. 
However,  these  indispensible 


instructor  functions  do  not  have  to 
be  carried  out  by  the  individuals  who 
serve  as  instructors  with  computer- 
based  systems.  In  fact,  it  would 
probably  be  more  cost-effective  for 
information  development  functions  to 
be  exclusively  the  jurisdiction  of 
professional  scientists,  scholars, 
computer  programmers,  instructional 
designers,  etc.  The  salience  of  this 
specialist  approach  to  information 
development  is  ramified  in  the  rapid¬ 
ly  emerging  areas  of  computer-based 
imagery,  animation,  simulation  gam¬ 
ing,  etc.  If  the  possibilities  for 
computer-based  instruction  are  to  be 
exploited  to  the  fullest  in  the  years 
to  come,  it  is  apparent  that  we  shall 
have  to  rely  more  upon  the  specialist 
(better  still,  teams  of  specialists). 

2.  It  is  in  the  performance  of 
information  delivery  functions  that 
the  computer-based  instructional 
system  proves  its  worth.  The  degree 
to  which  such  a  system  can  carry  out 
the  automated  functions  listed  pre¬ 
viously  depends  on  the  fidelity  of 
its  man-machine  interfacing  (sensors 
and  manipulanda;  visual,  auditory, 
etc.,  display  devices),  its  CPU 
capacity,  and  the  sophistication  of 
its  software.  If  we  wish  to  think 
very  far  ahead,  display  functions 
might  be  expanded  to  include  computer- 
driven  machine  movements  (robotics) . 

It  is  apparent  that  we  can  ex¬ 
pect  dramatic  advances  in  all  areas 
of  computer  technology  in  the  rela¬ 
tively  near  future.  However,  even 
current  capabilities  permit  the  exe¬ 
cution  of  nearly  all  information 
delivery  functions  for  which  human 
instructors  are  responsible,  includ¬ 
ing  some  functions  unachievable  by 
any  means  other  than  the  computer 
(e.g.,  interactive  video  displays). 
This  is  not  to  say  that  computerized 
machines  duplicate  human  actions  in 
carrying  out  the  same  functions,  or 
that  such  duplication  is  always  de¬ 
sirable  even  when  it  is  possible. 

For  example,  it  may  be  that  some 
information  car  be  transmitted  to  a 
student  more  effectively  in  a  writ¬ 
ten,  visual  display  than  in  a  spoken, 
auditory  display,  where  the  latter 
might  be  the  format  traditionally 
used  by  human  instructors.  Converse¬ 
ly,  if  it  were  beneficial  to  transmit 
the  information  in  a  simulated  human 
form,  a  speech  synthesizer  could  be 
employed.  Thus,  in  order  to  dupli¬ 
cate  human  functions,  it  may  not  be 
necessary  to  simulate  hunan  actions. 


A  review  of  the  capabilities 
listed  under  automated  functions 
indicates  a  high  order  of  individu¬ 
alized  and  interactive  information 
delivery  by  computer-based  instruc¬ 
tional  systems.  The  question  is,  are 
there  any  additional  delivery  func¬ 
tions  that  must  be  performed  by  a 
human  inst  r  uctor-c-Ferator  whose  sta¬ 
tion  is  a  terminal  tied  into  a  net¬ 
work  of  student  stations?  It  is 
implicit  in  this  question  that  the 
instructor's  communication  with  stu¬ 
dents  is  limited  to  just  those  chan¬ 
nels  available  in  his  station.  These 
may  be  of  two  types:  (a)  extra-CPU, 
and  (b)  intra-CPU.  Ey  means  of  the 
extra-CPU  channels,  the  instructor 
may  deliver  ncn-computer-based  infor¬ 
mation  (e.g.,  speech  and  other  accou- 
stic  signals,  visual  signals  and 
displays,  etc.).  These  displays  may 
be  presented  either  independently  of, 
or  in  synchrony  with,  CFU-produced 
information.  Also,  the  instructor 
may  selectively  open  channels  to  one 
or  more  student  stations,  enabling 
two-way  video  and/or  audio  communica¬ 
tion.  By  means  of  the  intra-CPU 
channels  the  instructor  may  interact 
in  ways  and  at  times  permitted  by  the 
program  (e.g.,  as  a  team  leader,  as  a 
competitor  in  a  gaming  situation,  as 
a  prompter,  etc.)  or  he  may  override 
the  program  and  initiate  appropriate 
pre-program  directives,  messages, 
data  listings,  etc. 

These  are  some  of  the  delivery 
functions  an  instructor  may  perform 
from  within  a  network  station.  Due 
to  this  system's  large  capability  for 
performing  automated  functions,  prob¬ 
ably  none  of  the  possible  human  in¬ 
structor  information  delivery  func¬ 
tions  can  be  considered  a  pr ior i  to 
be  indispensible.  Depending  on  spe¬ 
cific  training  program  requirements, 
both  extra-  and  intra-CPU  instructor 
functions  may  be  considered  desirable 
features  and  incorporated  as  part  of 
the  whole  delivery  system.  Since 
none  of  the  automated  functions  need 
be  sacrificed  in  order  to  accomplish 
this,  it  seems  highly  likely  that 
instructor  functions  of  the  sort 
indicated  above  would  be  included 
wherever  possible.  In  addition  to 
monitoring  students,  inclusion  of 
these  functions  would  enable  the 
instructor  to  interact  with  his  stu¬ 
dents  on  an  individual  or  group  basis, 
competing  with  or  leading  them,  cor¬ 
recting  or  encouraging  them,  di¬ 
recting  or  questioning  them,  etc. 

The  motivational  influences  of  these 


kinds  of  interactions  on  both  in¬ 
structors  should  magnify  the  effec¬ 
tiveness  of  the  entire  learning  ex¬ 
perience  in  this  situation. 

The  instructor's  responsibili¬ 
ties  for  delivery  information  outside 
of  the  network  station  probably  should 
be  regarded  as  essential.  They  would 
include  familiarization  of  students 
with  training  systems  operating  pro¬ 
cedures,  demonstrating  system  opera¬ 
tion,  orienting  students  to  training 
objectives  and  performance  expecta¬ 
tions,  carrying  on  post-training 
session  discussions  with  students, 
etc.  These  functions  would  probably 
require  that  the  instructor  serve  in 
several  traditional  roles:  as  lecturer 
during  initial  familiarization;  as 
supervisor  during  demonstration  of 
system  operations;  and  as  discussion 
leader  during  pre-  and  post-training 
sessions.  By  contrast,  during  system 
training  sessions  the  instructor's 
role  would  be  characterized  by  a 
mixture  of  elements  from  the  tradi¬ 
tional  roles  of  field  (team)  leader, 
supervisor,  and  tutor.  An  additional 
set  of  elements  of  a  non-traditional 
nature  may  also  be  included,  i.e., 
those  of  a  competitor.  The  compound 
of  these  elements  -esults  in  the 
emergence  of  a  new  role  for  the  in¬ 
structor  station.  For  want  of  a 
better  name  for  this  role,  we  shall 
call  it  "coach". 

3.  Student  supervision  is  a 
category  of  functions  which  ulti¬ 
mately  depend  on  the  delivery  of 
information  from  instructor  to  stu¬ 
dent.  Although  this  information  is 
instructional,  the  knowledge  compo¬ 
nents  of  it  are  generally  limited  to 
system,  or  machine,  operations  con¬ 
veyed  through  demonstrations  of  the 
skills  involved  in  performing  the 
operations.  Supervisory  information 
also  consists  of  evaluative  feedback 
from  the  instructor  regarding  both 
the  actions  made  in  skills  perfor¬ 
mance  and  the  outcomes  or  products  of 
those  actions.  Characteristic  of 
supervisory  information  is  a  strong 
directive  component  which  is  intended 
to  control  both  actions  and  outcomes. 
As  the  instructor's  responsibility 
for  supervisory  functions  depends  on 
how  closely  the  students'  actions  are 
to  be  controlled.  As  indicated  in 
the  list  of  automated  supervisory 
functions,  the  computer-based  system 
enables  a  high  order  of  student  di¬ 
rection  on  a  response-by-response 
basis.  The  instructor  is  thus 
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relieved  of  much  of  the  drudgery  of 
student  supervision,  this  being  re¬ 
placed  by  finely  attuned  immediate 
response-contingent  feedback  to  the 
student.  Supervisory  functions  re¬ 
maining  for  instructors  to  perform, 
as  in  the  case  of  information  de¬ 
livery  functions,  must  be  divided 
into  station  and  non-station  func¬ 
tions,  the  former  being  broken  down 
into  extra-  and  intra-CPU  functions. 

While  operating  from  his  network 
station,  the  instructor's  supervisory 
responsbilities  would  require  moni¬ 
toring  of  student  performance  via 
both  computer  and  non-computer  dis¬ 
plays.  Instructor  supervision  via 
intra-CPU  channels  might  consist  of 
program  overrides,  re-initializing 
some  portion  of  the  program,  inter¬ 
acting  with  students  in  ways  permit¬ 
ted  by  the  program  (e.g.,  entering 
instructions,  calling  up  visual  dis¬ 
plays,  manipulating  pointers  and  such 
on  visual  displays,  etc.),  activating 
pre-programmed  directives,  etc.  By 
means  of  extra-CPU  channels,  the 
instructor  might  issue  commands, 
instructions,  corrections,  etc.,  at 
any  point  independently  of  CPU-con¬ 
trolled  actions,  whether  or  not,  and 
the  degree  to  which,  any  of  these 
activities  were  considered  necessary 
would  depend  on  particular  program 
requirements.  The  instructor's  super¬ 
visory  duties  outside  of  his  network 
station,  and  after  initial  familiari¬ 
zation  and  demonstrations,  probably 
would  be  negligible  consisting  mainly 
of  informal  advice  and  encouragement. 
Thus  none  of  the  instructor's  super¬ 
visory  functions  can  be  considered  a 
priori  indispensable,  though  certainly 
they  would  be  desirable  in  most  cases, 
especially  those  involving  skill 
performance. 

It  is  interesting  to  observe 
that,  even  if  the  instructor's  super¬ 
visory  functions  are  strong,  in  this 
type  of  automated  instructional  situ¬ 
ation  his  role  would  not  be  that  of 
the  traditional  supervisor.  Rather, 
it  would  be  that  of  a  coach  as  indi¬ 
cated  earlier.  On  the  other  hand,  if 
supervisory  responsibilities  were 
negligible,  the  instructor's  role 
would  tend  toward  that  of  the  tutor 
(if  he  supplied  knowledge  information 
interactively)  or  that  of  the  com¬ 
petitor  (in  a  gaming  simulation). 

4.  The  advisory  and  counseling 
responsibilities  usually  subsumed 
under  student  guidance  probably  would 


be  largely  peripheral  to  the  instruc¬ 
tor's  network  station  functions.  The 
instructor's  responsibility  for  stu¬ 
dent  guidance  outside  of  the  network 
station,  however,  would  include  both 
formal  and  informal  discussions  re¬ 
garding  student  career  goals,  rele¬ 
vance  of  the  current  training,  to 
attainment  of  those  goals,  job  avail¬ 
ability  and  requirements  (some  of 
this  may  be  available  at  the  stu¬ 
dent's  network  terminal),  etc.  Fur¬ 
thermore,  specific  advisory  or  coun¬ 
seling  sessions  probably  would  be 
scheduled  to  correspond  with  major 
course  milestones.  At  such  times 
students  may  be  faced  with  decisions 
involving  changes  in  job  aspirations. 
The  instructor's  counseling  respon¬ 
sibilities  probably  should  be  limited 
to  providing  recommendations  for 
alternative  courses  of  action  for 
students  to  pursue  depending  on  their 
success  in  the  present  course  of 
instruction.  Formal  counseling  by 
instructors  might  be  carried  out  in 
conjunction  with,  or  referred  en¬ 
tirely  to,  a  professional  counselor 
knowledgeable  in  the  area  of  instruc¬ 
tion.  Thus,  the  computer-based  sys¬ 
tem  of  instruction  under  considera¬ 
tion  would  seem  to  permit  a  large 
range  of  instructor  commitment  to 
student  guidance  and,  consequently, 
this  may  not  be  regarded  as  an  indis- 
pensible  area  of  instructor  func¬ 
tions.  At  least  some  modest  level  of 
informal  counseling,  however,  would 
appear  to  be  beneficial. 

5.  As  indicated  in  the  list  of 
automated  student  evaluation  of  func¬ 
tions,  the  computer-based  system 
requires  little  additional  instructor 
input  in  this  area,  if  any  at  all. 
However,  if  the  instructor's  super¬ 
visory  responsibilities  within  his 
network  station  constitute  a  signifi¬ 
cant  element  in  the  training  program 
the  instructor's  responsibility  for 
some  forms  of  evaluative  feedback  to 
students  would  be  greater.  Such 
feedback  would  be  in  the  form  of 
corrections,  reprimands,  indications 
of  performance  deficiencies,  etc. 
Likewise,  because  the  system  can  be 
programmed  to  provide  the  student 
with  virtually  any  form  of  evaluative 
performance  index,  diagnoses  of  learn¬ 
ing  progress,  overall  performance 
profiles,  etc.,  the  instructor's 
responsibility  for  student  evaluation 
outside  of  his  network  station  would 
consist  largely  of  interpretation  and 
explanation.  Of  course,  need  for  the 
latter  would  depend  on  the  ease  with 
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which  students  could  read  and  inter¬ 
pret  their  machine-generated  evalua¬ 
tions.  In  this,  as  in  any  other  area 
of  computer-based  instruction,  pro¬ 
gramming  inadequacies  devolve  into 
instructor  responsbil ities.  In  sum¬ 
mary,  the  network  system  instructor’s 
student  evaluation  functions  are 
negligible  except  where  supervisory 
responsibilities  are  high.  In  that 
case,  the  two  areas  of  instructor 
functions  are  parallel  and,  to  some 
extent,  indistinguishable. 

6. -  In  the  area  of  data  manage¬ 
ment,  instructor  functions  are  elimi¬ 
nated.  Instructors  may  enter  their 
comments,  evaluative  scores,  etc., 
into  student  records,  but  the  in¬ 
structors  bear  no  responsibility  for 
filing,  updating,  or  other  processing 
of  performance  data. 

7.  The  mechanics  for  course 
management  are  all  pre-set  in  the 
training  program.  Instructor  respon¬ 
sibilities  in  this  area  consist  main¬ 
ly  of  keeping  the  prearranged  program 
on  schedule,  ensuring  smooth  inte¬ 
gration  of  non-program  activities 
with  the  schedule.  Thus,  insofar  as 
course  management  is  concerned,  the 
instructor's  role  is  essentially  that 
of  an  aide.  He  must  service  and 
otherwise  work  to  ensure  the  uninter¬ 
rupted  continuance  of  the  program. 

In  a  sense,  the  program  manages  it¬ 
self.  This  is  to  say  that  all  man¬ 
agement  decisions  regarding  schedul¬ 
ing,  resource  allocations,  program 
requirements,  contingency  plans  (in¬ 
cluding  provisions  for  instructor 
intervention),  etc.,  are  made  prior 
to  implementation. 

INSTRUCTOR  STATIONS 

The  kernel  of  our  analysis  is 
that,  if  computer-based  automated 
instructor  functions  are  utilized  to 
their  fullest  potential,  no  human 
instructor  functions  may  be  consider¬ 
ed  5  pr  ior  i  as  indispensible.  The 
machine  is  much  more  than  an  extra¬ 
ordinary  medium  for  the  delivery  of 
instructional  information.  It  is 
also  a  capable  instructor. 

The  question  becomes,  then,  what 
human  instructor  functions  may  be 
considered  desirable?  A  cursory 
enumeration  of  these  functions  was 
given  in  the  preceding  section.  They 
were  divided  into  two  situational 
classes;  functions  to  be  performed  in 
the  instructor’s  network  station,  and 


functions  to  be  performed  outside  of 
the  station.  Within-station  func¬ 
tions  were  further  divided  into  extra- 
CPU  channels  and  intra-CPU  channels. 
The  question  now  that  we  have  to 
address  is  what  characteristics  of 
network  instructor  stations  are  nec¬ 
essary  to  permit  the  performance  of 
human  instructor  functions? 

The  model  of  the  computer-based 
instructional  system  (trainer)  that 
we  are  considering  consists  of  a 
network  of  terminals  tied  to  each 
other  through  a  CPU  and  through  di¬ 
rect  circuits.  The  CPU  assumes  the 
lion's  share  of  responsibility  for 
presenting  information,  collecting 
performance  data,  data  management, 
interactive  processing  of  inputs  and 
outputs,  sequencing,  etc.,  depending 
on  the  ways  it  is  programmed,  the  CPU 
may  call  for  instructor  actions, 
decisions,  feedback  to  students, 
etc.,  thus  structuring  and  integra¬ 
ting  instructor  functions  into  the 
complex  flow  of  events.  If  the  pro¬ 
gram  involves  a  gaming  simulation, 
the  instructor  may  interact  with 
students  on  a  competitive  basis  as  an 
opponent  or  as  a  team  leader.  Fur¬ 
thermore,  through  the  CPU  the  in¬ 
structor’s  station  may  be  tied  into  a 
network  of  instructor  stations  there¬ 
by  placing  at  the  instructor's  dis¬ 
posal  a  generic  data  base  built  up 
through  common  experience  with  the 
system  or  originally  designed  for 
instructor  training. 

It  should  be  clear  that  the  core 
of  this  system  is  the  CPU,  not  the 
instructor.  Its  programming,  capa¬ 
city,  and  speed  determine  what  it  can 
process  (the  inputs  it  can  accept, 
the  decisions  it  can  make,  and  the 
displays  it  can  present) .  The  in¬ 
structor  station,  as  well  as  student 
stations,  are  the  peripheral  input 
and  output  interfaces  with  the  CPU. 
Excluding  direct  links  between  sta¬ 
tions,  the  specifications  for  station 
input  sensors  and  output  displays 
must  conform  to  CPU  capabilities.  At 
present  CPU  technology  appears  to  be 
more  advanced  than  sensing  and  dis¬ 
play  technology.  With  some  temerity 
we  suggest,  therefore,  that  anything 
which  can  be  sensed  in  analog  and 
converted  to  digital,  and  vice  versa, 
is  within  the  capabilities  of  exist¬ 
ing  CPU  technology  and  thus  can  be 
incorporated  into  student  and  in¬ 
structor  stations.  Ignoring  tech¬ 
nological  limitations  due  to  such 
problems  as  digital  conversion  of 
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analog  inputs  and  ouputs,  etc.,  let 
us  examine  some  desirable  input  and 
display  characteristics  of  instructor 
stations. 

Interface  features  of  the  in¬ 
structor  station  may  be  organized 
into  a  two-by-two  contingency  table. 
The  columns  of  this  table  designate 
channel  type  and  the  rows  designate 
interface  type.  The  two  channels 
are,  as  before,  intra-CPU  and  extra- 
CPU.  The  two  rows  are  controls  and 
displays.  Thus  we  have  controls  and 
displays  for  each  channel,  resulting 
in  four  categories  of  instructor 
station  features. 

1.  Intra-CPU  displays;  there 
are  two  classes  of  displays  within 
this  category:  (a)  monitors  and  (b) 
read-outs.  The  intra-CPU  monitor 
displays  provide  the  instructor  with 
duplicates  of  the  CPU  outputs  to 
student  stations.  These  would  in¬ 
clude  both  visual  displays  (CRT)  and 
auditory  displays  (headphones).  The 
student  station  being  monitored  would 
be  selectable  from  the  instructor 
station,  as  would  the  type  display. 
Since  these  displays  duplicate  those 
in  student  stations,  they  could  be 
used  by  the  instructor  while  partici¬ 
pating  in  game  simulations.  The 
read-out  displays  would  provide  the 
instructor  with  private  CPU  outputs 
thereby  enabling  him  to  receive  in¬ 
formation  from  the  computer  indepen¬ 
dently  of  that  available  at  student 
stations.  Status  reports  on  all  sta¬ 
tions,  data  files,  prompts  and  mes¬ 
sages,  etc.,  would  be  delivered  at 
these  displays. 

2.  Extra-CPU  displays:  these 
displays  permit  two-way  communication 
between  students  and  instructor. 

They  may  be  audio  (headphone)  and/or 
video  (television)  displays.  In 
addition  to  communication  between 
students  and  instructor,  these  dis¬ 
plays  could  also  be  used  as  extra-CPU 
monitors.  in  either  case,  student 
stations  would  be  selectable  by  the 
instructor,  as  would  open  channel 
communication  with  all  stations  (audio 
only) .  By  means  of  one  of  these 
displays  (probably  audio),  students 
would  be  able  to  initiate  communica¬ 
tion  with  the  instructor. 

3.  Intra-CPU  controls:  These 
instructor  response  interfaces  are  of 
two  types:  (a)  operator  and  (b)  mas¬ 
ter.  The  operator  controls  duplicate 
those  present  in  each  student  station. 
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They  permit  the  instructor  to  engage 
the  system  as  a  participant  (competi¬ 
tor  or  leader)  in  game  simulations, 
etc.  The  master  controls  enable  the 
instructor  to  address  the  CPU  inde¬ 
pendently  of  student  stations.  By 
means  of  the  master  control  terminal, 
non-automated  instructor  functions 
provided  for  in  the  computer  program 
could  be  carried  out. 

4.  Extra-CPU  controls:  these 
controls  include  all  switches,  knobs, 
levers,  etc.,  necessary  to  activate, 
select,  operate,  etc.,  any  non-CPU 
devices  or  displays.  Instructor 
controls  for  extra-CPU  displays  would 
be  included  in  this  category. 


Of  course,  specific  design  fea¬ 
tures  of  instructor  stations  would  be 
determined  by  the  same  sorts  of  de¬ 
terminants  that  were  shown  to  influ¬ 
ence  traditional  instruction.  In¬ 
deed,  the  entire  network  system  would 
be  impacted  by  these  factors.  There 
is,  however,  an  additional  element 
which  must  be  taken  into  account  in 
designing  the  instructor  station.  It 
is  the  CPU;  the  sophistication  of  its 
programming,  the  capacity  of  its 
memory,  and  the  speed  of  its  opera¬ 
tion. 


The  technology  exists  to  provide 
advanced  CPU  hardware  as  it  does  for 
controls  and  to  a  lesser  extent  for 
displays,  but  the  program  software 
has  to  be  created  for  each  applica¬ 
tion  of  this  technology.  We  suggest 
that  system  design  and  software  de¬ 
velopment  should  be  parallel  efforts, 
both  directed  toward  the  same  end,  an 
integrated  instructional  system.  If 
the  contributions  of  instructional, 
software,  and  hardware  designers  are 
weighted  equally  throughout  the  plan¬ 
ning  and  development  stages  of  com¬ 
puter-based  instructional  systems, 
then  the  instructor  station  concept 
may  be  fully  realized.  At  that  point 
we  will  see  the  new  instructor  roles 
emerge;  the  coach,  the  competitor, 
the  team  leader,  etc.  Whatever  kind 
of  animal  we  build,  let  us  design  it 
so  that  its  appendages  and  its  brain 
form  an  integrated  whole.  In  that 
the  instructor  will  be  but  one  func¬ 
tioning  organ. 
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INTRODUCTION 

The  emerging  Navy  Undergraduate  Pilot  (VTX) 
training  system,  designed  under  the  guide¬ 
lines  of  OMB  Circular  A-109,  Is  a  training 
system  composed  of  a  number  of  Integrated 
elements.  These  elements  are  the  aircraft, 
simulators,  training  management  system, 
academic  system,  and  an  Integrated  logistics 
system.  In  the  McDonnell-Douglas/Brltlsh 
Aerospace  VTX  system,  the  simulators  are  In 
reality  a  family  of  training  devices  which 
provide  a  systematic,  hierarchlal  method  of 
training  hands-on  learning  objectives.  The 
purpose  of  this  paper  Is  to  present  the 
front-end  analysis  procedures  which  were 
used  during  design  of  the  Instructor 
stations  for  the  family  of  VTX  training 
devices. 

PRELIMINARY  ANALYSIS 

The  VTX  training  system  (VTXTS)  development 
effort  centered  around  a  cost-effective 
method  to  meet  the  training  requirements  of 
Navy  jet  pilots  through  the  year  2010.  The 
VTXTS  Is  based  upon  the  requirement  to  train 
to  criterion  87  Navy-provided  objectives 
and  any  developed  during  the  ensuelng  ISO 
analysis.  Each  of  the  contractors  In¬ 
volved  In  the  VTX  procurement  was  required 
to  use  Instructional  Systems  Development 
(ISO)  principles  and  methodology  In  the 
design  of  the  system.  The  ISD  analysis  also 
provided  answers  to  problems  In  areas  of 
design-to-cost  and  life  cycle  cost  and  re¬ 
source  requirements.  Through  the  use  of  ISD, 
a  usable.  Integrated  system  was  created  Into 
which  the  family  of  training  devices  were 
fully  Interfaced.  This  paper  describes  the 
Mdown-1n-the-trenches“  procedures  that  were 
used  by  the  team  of  analysts  who  developed 
the  functional  specifications  for  the  VTXTS 
simulator  Instructor  Operating  Station. 
Figure  1  depicts  the  Mathetlcs  analytical 
process  which  lead  to  the  functional  spec¬ 
ifications  for  the  VTXTS  Instructor 
Stations.  The  right  hand  blocks  depict  the 
groups  or  personnel  who  provided  data  to  or 
received  data  from  the  study.  The  left  hand 
blocks  are  the  analytical  steps  used  during 
the  effort. 

Within  the  ISD  process,  considerable  work 
was  performed  as  part  of  the  process  of 
developing  functional  specifications  for  the 


Figure  1 

VTXTS  Simulator  Front-End  Analysis 


family  of  simulators.  The  ISD  steps  In¬ 
volved  were: 

.  Navy  Jet  Pilot  Warfare  Specialty 
Job  Analysis 

.  Navy  Jet  Pilot  Task  Analysis 
.  Objective  Hierarchy  Development 
.  Media  Selection 
.  Syllabus  Specification 
.  Course  Map  Development 

Of  the  above  steps  In  the  ISD  process,  the 
media  selection  and  syllabus  development 
Inputs  formed  the  base  on  which  the  analysis 
began.  The  media  selection  models  suggested 
three  types  of  devices,  an  Instrument  Flight 
Trainer  (IFT),  an  Operational  Flight  Trainer 
(OFT),  and  an  Aerial  Situation  Trainer 
(AST).  These  devices  and  their  associated 
capabilities  support  a  hierarchlal  acquisi¬ 
tion  (simple  progressing  to  complex)  of 
flight  skills.  Table  1  lists  the  types  of 
training  devices,  the  key  capabilities  and 
the  associated  pilot  training  stages. 
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TABLE  1 


Device 

Capabilities 

Training  Stages 

I  FT 

*  No  visual 

*  Flight  Support 

*  Platform  Motion 

*  FAM 

*  Flight  A  Ground  Modes 

*  BI 

*  Auxiliary  IOS 

*  RI 

•  CPT  mode 

*  ANAV 

OFT 

*  Large  field-of-view 

*  G-suit,  G-seat 

*  Interactive  Target 

lSli|:J  -  >.  * 

*  CGI  Visual 

AST 

*  Dynamic  Visual 

*  Dome  Visual 

*  G-seat/G-suit 

*  TACF 

buffet  systems 

*  GUNS 

*  Interactive  Target 

*  WEPS 

*  Auxiliary  IOS 

*  ACM 

Building  on  the  data  produced  during  the  ISO 
development  process,  Mathetics,  Inc.  per¬ 
formed  an  analysis  which  led  to  a  functional 
specification  for  the  instructor  operating 
stations  for  these  devices.  The  design  was 
to  be  specifically  oriented  toward  making 
the  instructor  a  teacher  rather  than  an 
operator. 

CONSIDERATIONS 

Initially  MATHETICS'  study  team  reviewed  the 
literature  on  prior  simulator  design  and 
applicable  human  factors  considerations. 
Although  many  studies  and  authors  con¬ 
tributed  to  the  data  base  for  this  study, 
two  works  were  particularly  valuable.  The 
first,  "The  Instructor  Pilot's  Role  in  Simu¬ 
lation  Training"  by  Dr.  John  P.  Charles 
(1978),  formed  the  basis  on  which  the  design 
algorithms  were  formed.  The  second, 
"Training  Device  Design:  Human  Factor's 
Requirements  In  the  Technical  Approach",  by 
Dr.  Alfred  F.  Smode  (1972),  provided  a 
wealth  of  information  pertaining  to  design 
considerations  and  presents  many  pertinent 
examples  of  human  factors  considerations  for 
design  of  simulators. 

This  concept  of  a  generic  Instructor  station 
for  all  of  the  training  devices  was  a 
desirable  goal  for  the  VTXTS  for  several 
reasons.  Including  cost  and  risk  reduction, 
commonality  of  hardware  and  software,  and  a 


reduction  in  instructor  training  require¬ 
ments.  The  Instructor  consoles,  although 
generally  common,  would  have  some  dif¬ 
ferences  due  to  the  varying  capabilities  of 
the  devices  and  therefore,  the  magnitude  of 
the  requirement  for  instructor  monitoring 
and  instructional  activities. 


The  study  of  all  factors  to  be  considered  in 
designing  a  generic  IOS  revealed  that  the 
instructor  console  should  have  a  number  of 
computer-assistance  features  designed  to  aid 
the  instructor.  These  features  were: 

•  Standardization  of  Training 
Preprogrammed  Tessons  which  would 
provide  a  series  of  standardized 
exercises  graduated  in  difficulty 
with  some  capability  for  instructor 
Intervention  in  prescribed  ways  to 
control  the  training  process.  This 
concept  includes  the  possible  manual 
override  of  defined  mission  events  to 
accommodate  students  who  perform 
below  expectation  for  a  given  exer¬ 
cise  and  for  insertion  of  additional 
related  events  in  an  exercise  to 
challenge  student  who  exceed  scenario 
requirements.  These  additions  and 
deletions  of  training  events  are  only 
allowed  if  the  preprogrammed  mission 
scenario  software  allows  manual 
intervention. 
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•  Computer  Assisted  Measurement  System 
Automated  evaluation  and  scoring  with 
objective  criteria  adjusted  to  the 
stage  of  Instruction  and  level  of 
difficulty  for  each  student.  The 
system  would  provide  error  Indica¬ 
tions  and  information  which  are  dis¬ 
played  at  the  Instructor  console. 

•  Automated  Moni tor  and  Control 
Capability 

Computer  drl ven  multi -format,  inter¬ 
active  CRT  displays  at  the  10S  which 
present  performance/error  and  sylla¬ 
bus  status  information  In  all 
relevant  modes  and  in  variable  for¬ 
mats  (both  alphanumeric  and  graphic). 

•  Records  of  Student  Performance 
Software  recording  of  student  perfor- 
mance  for  debrief.  Student  perfor¬ 
mance  information  (syllabus  event 
would  be  error,  and  summary  informa¬ 
tion)  available  on  demand  for  use 
during  and  after  a  training  exercise 
(for  both  debrief  and  for  record¬ 
keeping  purposes). 

The  first  of  these  computer  assisted  fea¬ 
tures  became  a  driving  factor  in  the  IOS 
design.  The  nature  of  the  Navy  Jet  Training 
Command  requires  that  specific  prerequisite 
knowledges  and/or  skills  must  be  acquired 
prior  to  the  follow-on  event.  Therefore, 
the  training  problem  in  the  Undergraduate 
Pilot  Training  (UPT)  context  is  best  placed 
under  software  control  where  the  instructor 
can  only  minimally  alter  the  training 
scenario.  This  allows  for  training  syllabus 
configuration  control  across  the  entire  Jet 
Training  Command,  in  that  the  learning 
objectives,  conditions  and  standards 
assigned  to  a  particular  lesson  cannot  be 
modified  by  the  instructor  who  thinks  he  has 
“a  better  way".  These  preprogrammed  lessons 
are  designed  in  such  a  way  as  to  specify  the 
applicable  instructional  features,  controls, 
and  strategies  which  may  be  employed  to 
teach  certain  objectives  maneuvers,  or 
flight  elements.  One  might  contend  that 
such  an  approach  would  take  the  Instructor 
"out  of  the  loop"  however,  it  is  believed 
that  such  an  approach  would  actually  enhance 
the  instructor's  capability  to  teach  rather 
than  operate  the  device. 

Heavy  programming  of  lessons  is  also  neces¬ 
sary  where  automated  performance  measurement 
Is  to  be  utilized.  One  cannot  alter  the 
training  scenario  and/or  lesson  methodology 
without  Impacting  the  results  from  the  per¬ 
formance  measurement  system.  Preprogrammed 
lessons  also  create  a  predictably  capable 
pilot  with  the  skills  necessary  for  entry 
Into  the  weapons  system  oriented  aircraft 
located  at  the  various  Fleet  Readiness 
Squadrons  (FRS).  Therefore,  a  conscious 


effort  has  been  made,  In  concert  with  ISO 
methodology,  to  design  out  the  flexibility 
of  past  simulators.  Designing  out  flex¬ 
ibility  means  designing  In  standardization. 
It  Is  believed  that  designing  out  flexibil¬ 
ity  would  also  reduce  cost  and  Increase 
operating  simplicity.  The  MATHETICS  study 
team  held  that  It  Is  possible  to  design  an 
IOS  which  has  no  trai ni ng  control s  other 
than  Interactive  CRTs  which  were  controlled 
by  a  package  of  training  software  designed 
specifically  to  enhance  the  Instructor's 
capability  to  train. 

The  last  consideration  of  note  was  the  con¬ 
cept  of  separating  the  simulation  system 
software  (aero  model,  motion  model,  etc.) 
from  the  package  of  training  software 
(syllabus,  scenario  control,  etc.)  In  terms 
of  configuration  control  and  management. 
Detailed  unique  training  software  programs 
can  be  created  for  each  lesson  containing 
the  initial  conditions  for  each  segment, 
allowable  instructional  features  and  strate¬ 
gies,  and  performance  measurement  routines. 
These  packages  of  training  software  would 
act  as  executive  routines  for  the  entire 
training  device  computational  system, 
prioritizing  programs,  routines  and  features 
in  terms  of  training  requirements.  The 
training  software  would  be  under  the  control 
of  a  Training  Support  Center  and  would  be 
easily  updatable  as  part  of  the  training 
system  quality  control  effort.  Instruc¬ 
tional  personnel  would  create  the  training 
methodologies  which  would  then  be  imple¬ 
mented  in  software.  This  concept  would 
allow  for  training  implementation  of  the 
syllabus  by  flight  instructors  trained  to 
instruct  in  the  visual/motor  skills  of 
flying  high  performance  jet  aircraft. 


ANALYSIS  PROCEDURES 

Based  upon  the  results  of  the  initial  data 
collection,  the  MATHETICS  study  team 
selected  the  Operational  Flight  Trainer 
(OFT)  as  the  first  trainer  for  analysis. 
This  was  done  primarily  because  the  OFT 
syllabus  encompassed  every  training  module 
within  the  VTXTS  curriculum  and  varied  in 
complexity  from  simple  hands-on  system 
familiarization  to  weapons  delivery.  The 
functional  specification  for  the  OFT  could 
then  be  later  adapted  for  use  in  the  Instru¬ 
ment  Flight  Trainer  (IFT)  and  the  Aerial 
Situation  Trainer  (AST).  The  MATHETICS 
study  team  selected  sixteen  lessons  from  the 
OFT  syllabus  for  full  development  and  analy¬ 
sis.  The  lessons  were  written  from  the 
existing  lesson  specifications  developed  in 
previous  project  work.  The  sixteen  lessons 
were  selected  for  their  instructional  con¬ 
tent,  the  nature  and  intensity  of  Instructor 
involvement;  and  the  possibility  for  use  of 
training  features  or  strategies. 
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Each  of  these  sixteen  lessons  was  then  fully 
analyzed  In  order  to  determine  the  sequence 
and  validity  of  the  learning  objectives, 
detail  Instructor  and  student  activities, 
and  estimate  the  scope,  possible  application 
of  simulator  learning  strategies  and  the  use 
of  instructional  features.  After  full  lesson 
development  the  analysts  looked  at  each  of 
the  lessons  for  a  number  of  attributes  and 
selected  six  of  the  original  sixteen  lesson 
scenarios  for  further  analysis.  The  scope 
of  the  general  training  requirements 
included: 

*  Basic  Instrument  Procedures 

*  Procedural  Demonstration 

Requirements 

*  Moduli  zed  Lessons 

*  Emergency  Procedures 

*  Verbally  Described  Demonstrations 

*  Self-Training  Requirements 

*  High  Density  Student/Instructor/ 

System  Interactions 

*  Visual  Task  Training 

*  Replay  requirements 

*  Performance  Measurement 

System  Requirements 

*  Backward  Chaining  Requirements 

*  Intense  Instructional  Activity 

*  Subjective  Instructor  Evaluation 


The  second  portion  of  the  analysis  utilized 
a  functional  flow  analysis  procedure  adapted 
from  the  work  of  both  Smode  and  Charles. 
The  functional  flow  analysis  involved  the 
development  of  "training  algorithms"  aided  • 
by  Subject  Matter  Experts  (SME).  The  SMEs 
were  asked  to  examine  each  of  the  lessons 
and  describe  how  they,  as  flight  instruc¬ 
tors,  would  best  teach  each  of  the  lessons. 
This  process  yielded  a  set  of  eleven 
training  algorithms  which  were  entitled: 

*  Pre-Briefing 

*  Lesson  Modification 

*  Briefing 

*  Initialization 

*  Start 

*  Demonstration 

*  Single  Task  Segnent 

*  Multiple  Repetition  Segment 

(loop  segnent) 

*  Post-Mission 

*  Debrief 

*  Post-Debrief 


The  analysts  then  created  strings  of  these 
algorithms  which  corresponded  to  the  lessons 
as  written  in  the  earlier  development 
effort.  The  Instructional  methods  and  in¬ 
structional  flow  for  each  of  the  six  lessons 
was  then  validated  by  SMEs.  These  strings 
of  algorithms  became  the  functional  flow 
diagram  for  each  lesson.  This  step  in  the 


process  really  involves  developing  an  over¬ 
view  of  the  Instructor's  role  In  the 
training  system  in  order  to  consolidate 
thinking  about  design  philosophy  and  aid  In 
the  further  definition  of  design  require¬ 
ments  for  the  instructor  station. 

The  next  step,  the  allocation  of  features, 
displays,  and  controls,  began  with  a  review 
of  the  literature  to  determine  the  types  of 
Instructor  Station  capabilities  which  were 
currently  being  utilized  in  existing  and 
near  term  simulators.  These  capabilities 
fell  into  seven  categories: 

*  Instructional  Features 

*  Displays 

*  Controls 

*  Console  Type 

*  Instructor  Activity  Type 

*  Controlling  System/Software 

*  Subsystem  Involvement 


A  "relative  time-line"  allocation  of  simula¬ 
tion  system  capabilities  was  then  performed, 
again  utilizing  heavy  SME  inputs.  The 
"relative  time-line"  analysis  allowed  speci¬ 
fication  of  activities,  instructional  fea¬ 
tures,  controls,  displays,  etc.  for  each 
activity  on  the  functional  flow  diagram.  An 
alphabetical  character  was  assigned  to  each 
of  the  above  listed  instructor/system  capa¬ 
bilities.  Each  subactivity  was  assigned  a 
numerical  identifier.  The  alphanumeric 
identifiers  were  placed  under  each  func¬ 
tional  flow  diagram  in  "relative  time-line" 
fashion.  This  data  format  can  be  seen  in 
Figure  2  which  depicts  a  portion  of  a  func¬ 
tional  flow  diagram  containing  a  single  task 
algorithm.  This  format  was  utilized  to 
determine  the  instructional  requirements  for 
each  lesson  and  to  provide  a  qualitative 
indication  of  frequency  of  use  and  an  indi¬ 
cation  of  criticality.  Additionally,  the 
functional  flow  analysis  permitted  the 
analysts  to  "visualize"  the  desired  sequence 
of  instructional  activities  and  options 
available  to  the  instructor  while  actually 
involved  in  a  training  evolution. 


SPECIFICATION  OF  INSTRUCTIONAL  FEATURES 
AND  STRATEGIES 

Based  upon  the  data  that  was  generated 
during  the  preceding  analyses,  a  set  of 
instructional  strategies  and  features  for 
the  simulator  was  developed.  Instructional 
strategies  are  defined  as  simulator  capa¬ 
bilities  which  are  utilized  as  methods  of 
presenting  simulator  training  materials. 
The  instructional  strategies  applicable  to 
the  VTXTS  OFT  are  DEMONSTRATION,  REPLAY, 
PREPROGRAMMED  LESSONS,  MANEUVER/SEGMENT 
RERUN,  BACKWARD  CHAINING  and  FREE  FLIGHT. 
These  strategies  are  defined  below. 
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DEMONSTRATION  is  an  OFT  Instructional  strat¬ 
egy  that  consists  of  a  prerecorded  aircraft 
maneuver,  or  series  of  maneuvers,  that  pro¬ 
vides  a  model  of  desired  performance  for  a 
flight  event.  REPLAY  enables  the  Instructor 
to  replay  the  student's  performance  during 
any  portion  or  all  of  the  most  recent  five 
minutes  of  simulated  flight.  PREPROGRAMMED 
LESSONS  comprise  98%  of  all  training  events 
and  consist  of  a  pre-defined  sequence  of 
maneuvers,  segments  of  flight,  Instructional 
events  and  allowable  instructional  features 
and  strategies  which  reside  In  the  OFT 
training  software.  MANEUVER/SEGMENT  RERUN 
Is  an  Instructional  strategy  which  permits 
the  Instructor  to  return  to  the  flight  con¬ 
ditions  which  existed  at  the  beginning  ot  a 
manuever,  either  by  flight  segment  designa¬ 
tion  or  by  entering  any  mission  time  within 
the  five  minute  replay  recording.  BACKWARD 
CHAINING  Is  a  strategy  which  presents  a  set 
of  prerecorded  points  (Initial  conditions) 
which  allow  a  student  to  learn  the  last 
response  (last  portion)  In  a  response  or 
maneuver  chain  first.  Learning  then  pro¬ 
ceeds  "backward"  up  the  chain  until  all 
members  of  that  chain  are  acquired.  FREE 
FLIGHT  lessons,  comprising  approximately  2% 
of  the  scenarios,  enable  the  Instructor  to 
construct  his  own  flight  scenario  from  a 
menu  of  Initial  conditions,  malfunctions, 
flight  segments,  and  Instruction  modules. 


Instructional  features  are  defined  as  simu¬ 
lator  capabilities  which  enhance  the 
training  of  a  student  by  modifying  the 
instructional  scenario  or  strategies.  The 


features  applicable  to  the  OFT  were  deter¬ 
mined  to  be  SELECTABLE  DIFFICULTY  LEVELS 
(SDL),  DIGITAL  SPEECH  GENERATION,  and 
PARAMETER  FREEZE.  SELECTABLE  DIFFICULTY 
LEVELS  is  an  OFT  instructional  feature  that 
consists  of  a  simultaneous  programming  of 
three  levels  of  difficulty  for  any  scenario 
and  which  Is  utilized  to  either  challenge  or 
unload  a  student  whose  performance  Is  above 
or  below  the  UPT  student  average  respec¬ 
tively.  SDL  is  In  reality  a  simple  "poor 
man's"  adaptive  training  capability. 
DIGITAL  SPEECH  GENERATION  unloads  the 
instructor  of  the  requirement  to  act  as  a 
manned  Interactive  system  (GCA  controller, 
ATIS,  clearance  delivery  ACLS  messages,  LSO 
control,  etc.)  by  means  of  speech  synthesis. 
PARAMETER  FREEZE  allows  the  Instructor  to 
selectively  freeze  a  certain  flight  para¬ 
meter  (airspeed,  angle-of-attack,  etc.)  or  a 
portion  of  a  flight  path  (glldeslope,  cen¬ 
terline,  etc.)  to  within  acceptable  perfor¬ 
mance  measurement  parameters. 

It  Is  felt  by  many  simulation  Industry 
observers  that  Information  about  the  In¬ 
tended  use  to  be  made  of  a  simulator's  In¬ 
structional  features,  If  made  available 
during  the  design  process,  could  be  used  to 
design  a  more  effective  vehicle  for 
training.  The  needed  Information  must  con¬ 
vey  to  the  designer  the  prospective  simula¬ 
tor  user's  concept  of  how  the  various  in¬ 
structional  features  are  to  be  employed 
during  simulator  instruction.  It  was 
decided  by  the  study  personnel  that  the  use 
of  Instructional  Feature  Design  Guides, 
adopted  from  Pohlman,  Isley  and  Caro  (1978), 
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would  best  accomplish  this  purpose.  The  VTX 
Instructional  Feature  Design  Guides 
accounted  for  learner  characteristics  and 
teaching  methodologies  appropriate  to  Under¬ 
graduate  Pilot  Training.  Each  of  the 
instructional  strategies  and  features  was 
written  up  in  an  individual  design  guide 
which  described  the  feature  and  how  it  would 
be  employed  by  the  simulator  instructor  to 
perform  coaching,  demonstration,  feedback 
and  instructional  functions  for  relatively 
unskilled  pilots. 

A  six-element  format  was  utilized  for  the 
instructional  feature  design  guides  and 
included  the  following  items: 

*  Feature  name 

*  Definition 

*  Purpose  and  Intended  Use 

*  Function  Description 

*  Concurrent  Events 

*  Feature  Diagram 

Figure  3  is  an  example  of  a  Feature 
Diagram. 


tackmrd  Chaining  -  F««tur«  OUgraa 


Example  of  a  Feature  Diagram 


SYSTEM  DEVELOPMENT 

A  survey  of  existing  and  emerging  simulator 
technology  was  conducted  in  order  to  incor¬ 
porate  ideas  and  technology  which  would 
Improve  the  instructor's  capability  to 
teach.  The  two  devices  which  most  dramati¬ 
cally  Influenced  the  IOS  design  process  were 
the  F-14  Instructor  Support  Station  and  the 
F-18  OFT,  Device  2F132.  These  two  devices 


incorporate  a  number  of  features  that  were 
deemed  appropriate  as  a  result  of  the  pre¬ 
viously  discussed  analyses.  These  capa¬ 
bilities  of  interest  are  as  follows: 

•  F-14  Instructor  Support  Station  (ISS) 

*  Interactive  CRT  Touchpanels 

*  Preprogrammed  Lesson 

*  Digital  Speech  Generation 

*  Performance  Measurement 

System 

•  F-18  OFT 

*  Display  System 

*  Interactive  CRT 

*  Procedures  Monitor 

*  Preprogramed  Missions 

Of  particular  interest  is  the  simplicity  of 
the  F-18  OFT  10S.  In  fact,  the  VTX  family 
of  training  devices  all  include  an  IOS  which 
is  very  much  similar  in  hardware  configura¬ 
tion  to  the  2F132.  However,  they  are 
markedly  different  in  software  design, 
operational  capability  and  syllabus 
implementation  from  the  F-18  OFT. 

The  analyses  indicated  that  it  would  be 
desirable  to  include  two  additional  sub¬ 
systems  into  the  simulator  design  to  further 
enhance  its  capability  to  meet  challenges 
the  training  system.  These  subsystems  were 
an  Automatic  Performance  Measurement  System 
(APMS)  and  a  Remote  Briefing  and  Debriefing 
Station.  The  APMS  would  provide  a  computer 
system  that  would  score  a  student's  perfor¬ 
mance  automatically  and  package  those  scores 
in  a  manner  which  would  be  useable  to  the 
instructor  in  real  time  and  during  the  sub¬ 
sequent  debrief.  The  APMS  programs  will 
compare  the  student's  performance  with  a 
definitive  criterion,  normalize  these  scores 
in  order  to  provide  peer  ratings,  determine 
student  errors,  and  suggest  remedial  strate¬ 
gies.  The  APM  system  is  designed  in  such  a 
way  that  performance  on  one  leg  or  segment 
does  not  affect  scoring  on  subsequent  legs 
except  that  the  possible  use  of  the  SDL 
feature  may  be  continued. 

The  Remote  Briefing  and  Debriefing  Console 
consists  of  an  isolated  station  interfaced 
to  both  the  simulators  and  the  Training 
Management  System.  This  console  can  be 
utilized  for  Simulator  lesson  preparation, 
pre-mission  briefing,  post-mission  de¬ 
briefing,  student  data  management  and 
training  management  functions.  The  intent 
of  the  Remote  Briefing  and  Debriefing  Capa¬ 
bility  is  two  fold;  one,  to  allow  briefs  and 
debriefs  to  be  held  in  an  area  more  com¬ 
patible  to  the  instructional  activities 
involved  in  pre-  and  post-mission  training 
than  the  simulator  itself,  and  two,  to 
increase  the  available  OFT  training  time  by 
freeing  the  device  from  the  briefing  and  de¬ 
briefing  activities. 
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The  analyst  team  then  used  the  functional 
flow  diagrams  to  validate  the  10S  design, 
ensuring  that  the  six  lessons  could  be 
taught  as  envisioned.  This  validation  pro¬ 
cess  was  Iterative  In  nature  as  the  IOS 
design  and  the  functional  flow  diagrams  were 
continually  changed  and  updated  until  a 
working  design  was  completed.  This  analyti¬ 
cal  step  Is  similar  to  design  "mock-up" 
review  and  ensures  that  the  instructor  can 
Instruct  from  the  instructor  station.  The 
final  step  was  documentation  of  the  actual 
functional  specifications. 

LESSONS  LEARNED 

The  front-end  analysis  leading  to  the  VTXTS 
IOS  functional  specifications  was  not 
without  pitfalls.  As  the  analysis  pro¬ 
ceeded,  It  was  necessary  to  work  around  a 
number  of  problems.  The  following  para¬ 
graphs  describe  the  most  important  lessons 
learned  from  the  Mathetics  study  effort. 

1.  Simulation  engi neers  are  hardware 
specification  oriented,  rt  was  dif¬ 
ficult  to  communicate  training  require¬ 
ments  to  the  engineers  due  to  the  lack 
of  hardware  definition.  Feature  design 
guides  describing  in  detail  the  use  of 
instructional  features  to  meet  the  UPT 
training  requirements  largely  obviated 
this  problem. 

2.  The  ISO  base  on  which  the  IOS  front-end 
analysis  i s  anchored  must "Be  complete. 
ConsideraBTe  effort  must  be  expended  on 
the  development  of  learning  objectives, 
conditions  and  standards  prior  to  media 
selection  and  lesson  specification. 
Improper  media  selection  and/or  poor 
lesson  specifications  could  lead  to 
functional  specifications  which  do  not 
address  actual  training  requirements. 

3.  Subject  Matter  Expert  (SME)  participa¬ 
tion  is  essential  duri~ng~Tesson  deve¬ 
lopment  and  the  functional  flow 
analysi s.  tf 1 thout  user  inputs  the 
analytical  effort  may  end  up  in  left 
field.  SME  input  injects  reality  into 
the  front-end  efforts. 

4.  Functional  flow  analyses  are  1  abor 
Intensive" and  must  be  iterative  in 
nature.  The  functionaT  flow  analysi s 
is  not  the  area  to  skimp  on  manpower. 
The  work  must  be  iterative  in  nature 
and  previously  developed  algorithms 
updated  as  the  diagrams  are  created. 

5.  The  analytical  "mock-up"  is  necessary 
to  validate  the  functional  flows. 
Performing  a  "mock-up"  review  with  SMEs 
identifies  weak  points  and  allows  for 
improvement  of  the  specification. 
Without  this  step,  the  functional  flow 


diagrams,  and  feature  allocations  will 
never  communicate  the  instructional 
requi rements . 

6.  Front-end  analysis  can  yield  signifi¬ 
cant  improvement  in'  user/manufacturer 
cooperation.  SlTF  inputs  before  the 
development  of  the  functional  specifi¬ 
cation,  if  properly  Implemented,  will 
reduce  the  adverse  Impact  of  Fleet 
Project  teams  later. 

7.  Wi thout  front-end  analysi s  "gold 
plating"  may  result.  This  front-end 
analysis  showed  that  existing  or  near- 
term  technology  was  adequate  for  the 
VTXTS  simulators  and  incorporation  of 
exotic  features  and  capabilities  was 
not  necessary. 


FINAL  COMMENTS 

A  number  of  conferences  have  addressed  the 
value  of  front-end  analysis  In  simulator 
instructor  station  design  in  theoretical 
terms.  This  paper  has  presented  an  example 
of  one  approach  to  performing  a  front-end 
analysis  leading  to  functional  specifica¬ 
tions  that  has  been  tried.  The  methodology 
utilized  in  this  study  is  not  original  but 
is  adapted  from  previously  published  work. 
This  front-end  analysis  achieved  a  much 
better  collaboration  with  project  engineers 
than  ordinarily  occurs  during  simulator  IOS 
design  efforts  and  thus  greatly  enhanced 
their  understanding  of  the  UPT  training 
requirements  and  how  a  typical  scenario 
would  be  taught  on  the  device.  The 
Mathetics  study  team  sees  the  actual 
development  of  working  specifications  for  a 
simulator  IOS  as  an  iterative  process  which 
continues  throughout  the  simulator  procure¬ 
ment  cycle.  The  functional  specifications 
generated  as  a  result  of  this  front-end 
analysis  was  an  initial  training  input  to 
that  iterative  process.  The  training  team 
must  remain  involved  throughout  the 
engineering  design,  construction,  and  user 
acceptance  processes  in  order  to  ensure 
incorporation  and  proper  implementation  of 
key  training  recommendations.  Without 
training  requirements  inputs  and  analysis, 
there  is  a  risk  of  user  dissatisfaction  with 
the  device  when  placed  in  the  actual 
training  situation. 
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FOR  FIGHTER  AND  TRAINER  AIRCRAFT  FLIGHT  SIMULATORS 
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ABSTRACT 

A  conceptual  design  for  a  comprehensive 
Instructional  support  system  (ISS)  for  fighter 
and  trainer  aircraft  flight  simulators  was  de¬ 
veloped  as  part  of  an  Independent  research 
project  to  develop  and  evaluate  advanced  pilot 
training  techniques.  The  major  elements  of 
this  ISS  are: 

1.  A  low-workload  flight  simulator  in¬ 
structor  console  based  in  part  on  the  task 
module  approach  developed  by  Logicon  and  the 
Canyon  Research  Group  for  the  F-14  Operational 
Flight  Trainer  ISS; 

2.  A  comprehensive  automated  performance 
monitoring  system  based  on  techniques  devel¬ 
oped  by  Northrop,  Vought,  and  Systems  Technol¬ 
ogy,  Inc.  (STI) 

3.  A  set  of  automated  and  nonautomated 
simulator  instructional  features  selected  on 
the  basis  of  a  review  of  the  literature  and 
recent  field  experience 

4.  A  low-cost,  low-workload  brief/debrief 
console  designed  to  minimize  the  time  spent  in 
the  simulator  on  tasks  other  than  active 
instruction. 

These  elements  are  designed  to  work  together 
to  reduce  instructor  workload  and  increase  the 
effectiveness  and  efficiency  of  simulator 
training.  Although  the  design  is  oriented  to¬ 
wards  large,  highly-structured  training  sys¬ 
tems,  many  of  its  features  appear  applicable 
to  other  training  environments. 

INTRODUCTION 

A  conceptual  design  for  a  comprehensive 
instructional  support  system  (ISS)  for  fighter 
and  trainer  aircraft  flight  simulators  was  de¬ 
veloped  as  part  of  an  independent  research  and 
development  project  to  develop  and  evaluate 
advanced  pilot  training  techniques.  The  major 
elements  of  the  ISS  are  1)  a  low-workload  in¬ 
structor  console  based  in  part  on  the  task 
module  approach  developed  by  Logicon  and  the 
Canyon  Research  Group  for  the  F-14  operational 
flight  trainer  ISS;  2)  a  comprehensive  auto¬ 
mated  performance  monitoring  system  based  on 
techniques  developed  by  Northrop  for  evaluat¬ 
ing  student  performance  in  introductory  air- 
to-air  tactics,  by  Vought  for  evaluating  stu¬ 
dent  performance  in  carrier  landing  and  by  STI 
for  examining  pilot  control  techniques  in  a 
variety  of  tasks;  3)  a  set  of  nonautomated  and 
automated  simulator  instructional  features  se¬ 
lected  on  the  basis  of  a  review  of  the  recent 


literature  on  instructional  features,  supple¬ 
mented  by  interviews  with  research  and  user 
personnel;  4)  a  low-cost,  low-workload  brief/ 
debrief  console  designed  to  reduce  time  spent 
in  the  simulator  on  tasks  other  than  active 
instruction. 

Working  together  these  elements  appear  ca¬ 
pable  of  dramatically  reducing  instructor 
workload  and  significantly  increasing  the  ef¬ 
fectiveness  and  efficiency  of  simulator  train¬ 
ing.  In  general,  it  appears  that  the  proposed 
approach  should 

1.  reduce  the  instructors  perceptual  and 
procedural  workload  far  below  that  of  current 
simulators; 

2.  reduce  the  time  required  to  achieve  a 
given  level  of  student  performance; 

3.  permit  a  much  higher  level  of  training 
standardization  and  control  than  is  possible 
with  current  simulators; 

4.  provide  a  full  capability  for  tailor¬ 
ing  training  to  the  needs  of  individual 
students; 

5.  reduce  instructor  training  require¬ 
ments;  and 

6.  minimize  the  need  to  refer  to  user's 
manuals. 

DESIGN  GUIDELINES 

Two  basic  design  guidelines  were  estab¬ 
lished  for  this  development.  The  first,  a 
point  often  stressed  by  Mr.  William  Harris  of 
the  Analysis  and  Design  Branch  at  NTEC,  is 
that  the  design  of  all  equipment  to  be  used  by 
military  flight  instructors  should  reflect  the 
fact  that  they  are  typically  undermanned  and 
already  working  long  hours.  Their  time,  as 
well  as  their  energy  and  patience,  should 
therefore  be  treated  as  a  scarce,  essential 
resource  to  be  conserved  wherever  possible. 

It  should  be  recognized  for  example  that  in¬ 
structors  do  not  have  time  to  stop  in  the  mid¬ 
dle  of  a  training  session  to  try  to  remember  a 
complex  procedure  or  a  set  of  parameter  values 
in  order  to  set  up  a  given  maneuver.  They 
have  even  less  time  to  look  up  the  correct 
procedure  or  information  in  a  user's  manual. 
Instructor  consoles  and  other  equipment  to  be 
used  by  instructors  should  thus  be  as  simple, 
self-explanatory,  and  easy  to  use  as  possible. 
Achieving  such  simplicity  not  only  reduces  the 
time  normally  spent  in  remembering,  looking  up 
and  performing  procedures,  but  also  reduces 


set>up  errors  which  are  time  consuming  and 
frustrating  In  themselves.  Finally,  equipment 
that  is  simple  to  operate  should  significantly 
reduce  Instructor  training  requirements,  or  at 
least  allow  more  time  to  be  spent  on  teaching 
Instructors  how  to  instruct  rather  than  how  to 
operate  the  equipment. 

The  problem  with  all  this  is  that  making 
operating  procedures  simple  and  self-evident 
for  something  as  complex  as  a  flight  simulator 
typically  requires  the  development  of  fairly 
complex  software  to  partially  automate  the 
procedures.  The  development  of  such  software 
is  both  time-consuming  and  expensive.  Al¬ 
though  it  seems  clear  that  simpler  procedures 
can  contribute  to  reduced  life  cycle  costs  in 
several  ways,  there  is  simply  not  enough  data 
at  this  point  to  prove  that  the  projected  sav¬ 
ings  will  be  enough  to  offset  the  cost  of 
developing  and  maintaining  the  additional 
software.  A  second  and  very  important  guide¬ 
line  established  for  the  study  was  thus  the 
working  assumption  that  the  cost  of  developing 
and  maintaining  the  software  required  to  sim¬ 
plify  operator  procedures  will  be  offset  in 
the  long  run  by  savings  in  investment  and  op¬ 
erating  costs  resulting  from  such  benefits  as 
reduced  instructor  training  requirements  and 
more  effective  and  efficient  student  training. 
For  example,  the  proposed  instructor  console 
design  appears  capable  of  reducing  the  time 
now  spent  in  deciding  what  maneuvers  to  train, 
in  obtaining  needed  information,  and  in  set¬ 
ting  up  the  maneuvers.  These  savings  should 
translate  into  reduced  time  in  the  simulator 
for  each  student.  This  in  turn  should  result 
in  a  reduction  in  the  total  number  of  simula¬ 
tors  required,  and/or  a  reduction  in  the  op¬ 
erating  and  maintenance  costs  per  simulator. 

An  example  of  the  problems  caused  by  not 
simplifying  operator  procedures  is  provided  by 
some  current  military  instructor  consoles  that 
are  reported  to  require  the  instructor  to  per¬ 
form  as  many  as  40  procedural  steps  to  set  up 
a  single  maneuver  for  training.  Obviously 
such  a  design  will  waste  a  great  deal  of  stu¬ 
dent  time,  instructor  time,  and  device  time  in 
each  of  thousands  of  training  sessions  through¬ 
out  its  life  cycle.  Lengthy  set-up  procedures 
lower  simulator  training  efficiency  by  reduc¬ 
ing  the  ratio  of  productive  training  time  to 
total  simulator  time.  They  further  lower 
efficiency  and  effectiveness  by  introducing 
continual  delays  in  training  which  have  an  un¬ 
avoidable  effect  on  student  and  instructor 
concentration  and  motivation. 

INSTRUCTOR  CONSOLE 

Design  Approach 

A  high  level  of  partial  automation  is  used 
to  achieve  a  very  low  level  of  workload  for 
the  most  frequent  case  of  routine  or  standard¬ 
ized  training.  A  moderate  level  of  automation 


is  used  to  achieve  a  relatively  low  level  of 
workload  for  the  less  frequent  case  of  parti¬ 
ally  specialized  training.  A  low  level  of 
automation  is  used  for  the  relatively  rare 
case  of  fully  specialized  training,  resulting 
in  a  level  of  workload  for  this  type  of  train¬ 
ing  which  approaches  that  found  on  current 
consoles. 

The  application  of  automation  to  the  sim¬ 
plification  of  console  procedures  is  thus  in 
accordance  with  a  frequency-of-use  approach  in 
that  the  size  of  the  development  effort  to 
simplify  procedures  is  proportional  to  how  of¬ 
ten  the  procedures  are  used.  This  results  in 
a  console  that  is  "managed  by  exception"  in 
that  the  instructor  has  to  do  very  little  un¬ 
less  he  wishes  to  deviate  from  the  standard 
“default"  condition.  From  a  different  stand¬ 
point,  the  console  design  makes  use  of  the 
principle  of  least  effort  in  a  way  that  should 
result  in  a  much  higher  level  of  training  stan¬ 
dardization  than  is  realized  with  current  sim¬ 
ulators.  For  example,  the  console  design 
makes  it  easiest  of  all  for  the  instructor  to 
conduct  the  standard  training  specified  in  the 
syllabus  and  lesson  plan;  harder  but  still 
relatively  convenient  to  conduct  partially 
specialized  training;  and  least  convenient  of 
all  to  conduct  fully  specialized  training. 

Thus,  the  more  the  instructor  deviates  from 
the  syllabus  or  lesson  plan,  the  harder  he  has 
to  work.  In  this  way  the  design  strongly  en¬ 
courages  standardized  training  but  makes  it 
possible  for  the  instructor  to  introduce  spe¬ 
cialized  training  whenever  he  decides  that  it 
is  necessary. 

Physical  Configuration 

The  general  physical  configuration  of  the 
instructor  console  consists  of  an  interactive 
control/display  module,  a  secondary  display 
module,  and  a  two-axis  controller,  as  shown  in 
Figure  1.  The  interactive  module  consists  of 
a  21"  calligraphic  CRT  with  a  transparent 
membrane-type  on-screen  touch  panel  or  "touch 
screen",  and  several  off-screen  membrane-type 
illuminated  touch  switches,  where  individual 
switches  are  created  from  a  continuous  matrix 
of  switches  by  making  cutouts  where  desired  in 
a  thick,  e.g.,  1/8",  plastic  overlay.  The 
secondary  display  module  consists  of  a  second 
21"  calligraphic  CRT  plus  a  smaller  CRT  in¬ 
cluded  primarily  for  the  purpose  of  presenting 
the  instructor  with  the  view  seen  on  and 
through  the  students  HUD. 

The  configuration  shown  in  Figure  1  was 
designed  for  use  with  an  instrument  flight 
trainer  (IFT),  operational  flight  trainer 
(OFT),  or  air  combat  maneuvering  trainer 
(ACMT).  The  interactive  display  module  by  it¬ 
self  can  be  used  as  the  instructor  console  for 
a  cockpit  procedures  trainer  (CPT),  where  it 
can  be  swiveled  to  face  the  student  for  self- 
instruction.  For  an  IFT  or  OFT,  the  console 
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FIGURE  1.  GENERAL  CONFIGURATION  OF  VTXTS  SIMUL  TOR  INSTRUCTOR  CONSOLE 

(USED  ON  IFT,  OFT,  ACMT) 


is  located  in  back  of  and  to  the  left  of  the 
student's  seat,  and  at  right  angles  to  the  in¬ 
strument  panel.  An  instructor  seated  at  the 
console  can  thus  see  the  student  and  his  in¬ 
strument  panel  and/or  the  simulated  outside 
visual  scene  by  looking  over  his  right  shoul¬ 
der.  For  an  ACMT,  a  full  console  is  located 
outside  the  simulator,  and  a  miniconsole  con¬ 
sisting  of  an  interactive  display  module  is 
located  in  front  of  a  jumpseat  next  to  the 
cockpit. 

The  configuration  of  the  interactive  mod¬ 
ule  is  presented  in  Figure  2.  The  switches  on 
the  interactive  module  consist  of  dedicated 
special  function  switches  and  the  CRT  touch 
screen,  which  is  used  in  conjunction  with  var¬ 
ious  types  of  menu  displays  presented  on  the 
CRT.  The  selection  of  membrane-type  off¬ 
screen  touch  panels  over  mechanical  switches 
and  of  the  transparent  membrane-type  touch 
screen  over  infrared  and  sonar-type  touch 
screens  was  made  on  the  basis  of  a  detailed 
comparison  of  the  ease-of-use,  initial  cost, 
reliability  and  maintainability  of  these  al¬ 
ternatives.  A  caution  In  making  these  kinds 
of  control  hardware  comparisons  is  that  the 
cost  of  the  decoding  and  Interface  equipment 
required  for  different  types  of  switches  and 
switch  panels  is  often  much  greater  than  the 
cost  of  the  switches  or  panels  themselves,  and 
thus  must  be  broken  out  separately. 


FIGURE  2.  INTERACTIVE  CONTROL /DISPLAY  MODULE 


Functional  Design 

The  functional  design  of  the  instructor 
console  consists  of  several  major  features 
which  appear  capable  of  dramatically  simplify¬ 
ing  procedures  and  improving  training.  These 
features  are  as  follows: 

1.  An  extension  of  the  task  module  con¬ 
cept  developed  by  Logicon  and  the  Canyon 
Research  Group  for  the  F-14  OFT  ISS.  (Semple, 
et.  al.,  1979) 

2.  The  use  of  "event"  modules  in  addition 
to  task  modules. 

3.  The  inclusion  of  explicit  provisions 
for  conducting  three  basic  types  of  simulator 
training:  routine;  partially  specialized;  and 
fully  specialized. 

4.  The  inclusion  of  explicit  provisions 
for  controlling  four  basic  types  of  simulator 
learning  events:  IP  demonstration;  autopilot 
or  canned  demonstration;  live  run;  and  replay. 

5.  A  special,  minimal  set  of  dedicated 
function  switches  and  switch  logic  designed  to 
interact  with  event  modules,  task  modules  and 
special  purpose  displays  in  a  way  that  permits 
highly  efficient  control  of  each  type  of 
training  and  each  type  of  learning  event. 

6.  Low-density  pictorial  and  tabular 
special-purpose  displays  designed  to  provide 
the  instructor  with  the  essential  information 
at  each  point  in  the  training  process. 

The  following  sections  describe  the  na¬ 
ture,  use,  and  projected  advantages  of  each  of 
the  above  instructor  console  design  features. 

Task  Modules.  As  originally  conceived  by 
Logicon  and  the  Canyon  Research  Group  for  the 
F-14  OFT  ISS,  a  task  module  is  a  detailed  pre¬ 
specification  of  the  initial  conditions,  per¬ 
formance  conditions,  and  automated  performance 
measurement  algorithms  appropriate  to  a  given 
maneuver  or  task,  which  is  stored  in  computer 
memory.  One  of  the  primary  functions  of  such 
modules  is  to  automate  maneuver  set-up.  Using 
this  approach  an  instructor  can  set  up  a  simu¬ 
lator  to  train  any  given  maneuver  by  simply 
identifying  the  name  of  the  maneuver  to  the 
computer.  The  computer  then  sets  up  the  simu¬ 
lator  automatically  according  to  the  speci¬ 
fications  in  the  task  module.  Task  modules 
thus  greatly  reduce  instructor  workload  by 
eliminating  the  need  to  remember  or  look  up 
and  then  manually  enter  all  of  the  information 
which  must  be  specified  to  set  up  a  simulator 
to  train  a  specific  maneuver. 

The  proposed  approach  expands  the  original 
task  module  concept  in  two  ways.  First,  the 
task  module  specification  is  expanded  to  in¬ 
clude  additional  information  such  as  1)  the 


method  to  be  used  to  control  the  second  air¬ 
craft  in  ACM  and  formation  training;  2)  the 
standard  and  optional  instructor  console  dis¬ 
plays  to  be  shown  to  the  instructor  in  each 
part  of  each  maneuver;  and  3)  the  displays  to 
be  recorded  on  videotape  for  later  use  in  de¬ 
briefing.  Second,  "auxiliary"  task  modules 
are  created  which  enable  the  instructor  to 
quickly  set  up  standard  variations  on  basic 
maneuvers  in  order  to  tailor  training  to  the 
needs  of  individual  students  -  i.e.,  to  con¬ 
duct  partially  specialized  training  as  dis¬ 
cussed  below.  This  is  done  with  a  minimum 
impact  on  computer  memory  requirements  or 
software  development  costs  by  using  auxiliary 
modules  to  specify  only  those  parameter  values 
which  are  different  from  those  in  the  "core" 
module  for  the  basic  maneuver.  Standard  var¬ 
iations  on  a  given  maneuver  can  then  be  set  up 
by  selecting  a  desired  maneuver  variation  from 
a  menu  of  standard  variations.  In  addition  to 
greatly  reducing  instructor  workload,  core  and 
auxiliary  task  modules  will  increase  training 
standardization  to  a  level  not  possible  with 
current  consoles,  in  that  all  maneuvers  and 
maneuver  variations  will  be  set  up,  displayed 
to  the  instructor,  evaluated,  and  recorded  in 
the  same  way  for  each  student  and  instructor. 

The  procedures  for  fully  specialized 
training  are  simplified  by  requiring  the  in¬ 
structor  to  manually  input  only  those  parame¬ 
ter  values  which  are  different  from  those  in 
the  core  or  auxiliary  task  module  under  con¬ 
sideration,  as  discussed  below. 

Event  Modules.  Event  modules  are  an  ex¬ 
tension  of  the  canned  mission  mode  defined  for 
the  F-14  OFT  ISS.  As  defined  here,  an  event 
module  is  a  detailed  prespecification  of  the 
lesson  plan  for  a  given  training  event  which 
is  stored  in  computer  memory.  An  event  module 
is  created  for  each  simulator  event  in  the 
flight  syllabus.  The  event  module  for  a  given 
event  specifies  the  specific  maneuvers  to  be 
trained  in  the  event,  as  well  as  the  sequence 
in  which  they  will  be  trained.  It  also  spec¬ 
ifies  the  type  and  number  of  demonstrations  to 
be  used  in  the  training  of  each  maneuver,  as 
well  as  the  minimum,  standard,  and  maximum 
number  of  student  practice  trials  to  be  flown. 
The  use  of  a  prespecified  minimum,  standard, 
and  maximum  number  of  trials  will  be  further 
discussed  below  under  controls  and  displays. 

In  general  the  event  module  will  automatically 
set  up  the  simulator  to  fly  each  run  of  each 
maneuver  in  the  sequence  called  for  by  the 
lesson  plan  for  that  event.  It  will  do  this 
by  calling  up  appropriate  task  modules.  For 
example,  at  the  start  of  a  training  session 
the  event  module  will  automatically  set  up  the 
simulator  for  the  first  run  of  the  first  ma¬ 
neuver.  This  might  be.  e.g.,  an  automated 
demonstration  of  a  lazy  8.  After  he  finishes 
talking  to  the  student  and  verifies  that  the 
student  is  ready,  the  instructor  starts  the 
run  by  pushing  the  START  switch.  The  run  will 


stop  when  the  instructor  pushes  the  STOP 
switch  or  when  the  run  time  reaches  some  pre¬ 
specified  maximum  stored  in  the  task  module 
for  that  maneuver.  When  the  run  stops,  the 
event  module  will  start  setting  the  simulator 
for  the  next  run  to  be  flown,  typically  a  stu¬ 
dent  practice  run  ("live  run")  for  the  same 
maneuver.  After  the  standard  number  of  stu¬ 
dent  practice  runs  have  been  flown  for  the 
first  maneuver,  the  event  module  will  start 
setting  up  the  simulator  to  fly  the  first  run 
of  the  second  maneuver.  This  procedure  is  re¬ 
peated  until  all  maneuvers  have  been  flown. 
Unless  he  wants  to  deviate  from  the  standard 
lesson  plan,  the  only  inputs  the  instructor 
has  to  make  are  to  push  POSITION  SIM  to  ini¬ 
tialize  each  run,  START  to  start  each  run,  and 
NEXT  MANEUVER  to  proceed  to  the  next  maneuver. 

Although  it  may  appear  at  first  that  the 
event  modules  are  too  restrictive,  a  variety 
of  ways  are  provided  to  permit  the  instructor 
to  deviate  from  the  standard  lesson  plan  to 
tailor  training  to  the  needs  of  the  individual 
student.  As  with  task  modules,  the  difficulty 
of  deviating  from  the  standard  lesson  plan  for 
an  event  is  inversely  proportional  to  the  es¬ 
timated  frequency  of  the  need  for  such  de¬ 
viation.  Thus,  modifying  the  number  of  trials 
within  the  prespecified  minimum  and  maximum 
number  of  trials  is  very  easy.  Modifying  the 
lesson  plan  to  provide  training  on  a  maneuver 
which  is  not  even  in  the  lesson  plan  is  rela¬ 
tively  complex,  although  still  fairly 
straightforward. 

On  the  surface  it  may  seem  desirable  to 
let  a  student  practice  each  maneuver  or  task 
until  he  reaches  the  specified  or  customary 
level  of  performance  for  that  point  in  train¬ 
ing,  regardless  of  the  number  of  trials  called 
for  in  the  syllabus  or  lesson  plan.  In  prac¬ 
tice,  however,  most  large-scale  training  pro¬ 
grams  permit  only  slight  deviations  from  the 
planned  number  of  trials  for  each  maneuver. 
This  is  due  to  the  fact  that  large  deviations 
have  a  chain  reaction  effect  on  subsequent 
syllabus  events.  For  example,  excessive  time 
spent  on  one  maneuver  will  usually  result  in 
insufficient  practice  on  one  or  more  of  the 
other  maneuvers  scheduled  for  the  same  event. 
The  student  is  then  faced  with  the  problem  of 
going  to  the  next  regularly  scheduled  simu¬ 
lator  or  aircraft  training  event  without  being 
able  to  perform  these  other  maneuvers  at  the 
expected  level.  This  is  yet  another  example 
of  how  individualized  progression,  although 
desirable  from  a  theoretical  standpoint, 
creates  serious  problems  in  large  scale  train¬ 
ing  systems.  Recent  advancements  in  dynamic 
scheduling  techniques  may  someday  provide  an 
answer  to  this  dilemma. 

Provisions  for  Routine  vs  Specialized 
Training.  The  proposed  design  makes  explicit 
provisions  for  three  types  of  training:  rou¬ 
tine  training;  partially  specialized  training; 
and  fully-specialized  training. 


In  the  context  of  individual  maneuver 
training,  routine  training  consists  of  train¬ 
ing  the  student  to  perform  one  of  the  standard 
versions  of  the  maneuver  called  for  in  the 
flight  syllabus.  Partially  specialized  train¬ 
ing  in  this  context  refers  to  tailoring  train¬ 
ing  to  the  needs  of  the  student  by  having  the 
student  fly  a  relatively  common  variation  of 
one  of  the  standard  versions  of  the  maneuver, 
e.g.,  under  different  visibility  conditions  or 
with  a  different  c.g.  Fully  specialized 
training  refers  to  training  the  student  to  fly 
uncommon  variations  on  one  of  the  standard 
versions  of  the  maneuver,  e.g.,  in  turbulence 
with  one  engine  out,  etc. 


In  the  context  of  the  training  of  an  en¬ 
tire  event,  routine  training  refers  to  con¬ 
ducting  the  training  exactly  as  specified  in  a 
pre-established  lesson  plan,  in  terms  of  the 
maneuvers  trained,  the  sequence  of  maneuvers 
trained,  the  type  of  runs  or  "learning  events" 
used  to  train  each  maneuver  and  the  number  of 
runs  of  each  type  flown.  Partially  special¬ 
ized  training  here  refers  to  making  relatively 
slight  variations  in  the  lesson  plan,  e.g., 
varying  the  number  of  student  practice  runs 
for  a  particular  maneuver  within  pre- 
established  minimum  and  maximum  limits.  Fully 
specialized  training  here  refers  to  making  ma¬ 
jor  changes  in  the  lesson  plan  such  as  exceed¬ 
ing  the  maximum  number  of  runs  specified  for  a 
given  maneuver  or  introducing  a  maneuver  which 
is  not  in  the  lesson  plan. 


As  discussed  above,  the  console  is  de¬ 
signed  so  that  routine  training,  which  should 
be  completely  adequate  most  of  the  time,  re¬ 
quires  an  absolute  minimum  of  instructor  in¬ 
puts.  Partially  specialized  training,  needed 
occasionally,  requires  a  moderate  number  of 
instructor  inputs.  Fully  specialized  train¬ 
ing,  needed  only  on  rare  occasions,  requires  a 
relatively  high  number  of  manual  inputs.  In 
comparison  with  current  consoles  instructor 
workload  should  thus  be  very  low  for  routine 
training,  higher  but  still  relatively  low  for 
partially  specialized  training,  and  approach 
that  of  current  consoles  only  for  the  rela¬ 
tively  infrequent  case  of  fully  specialized 
training. 


Provisions  are  also  made  to  record  the 
number  and  nature  of  each  partially  or  fully 
specialized  maneuver  or  task  used  by  the  in¬ 
structor  to  identify:  1)  standard  variations 
of  maneuvers  and  tasks  which  are  used  fre¬ 
quently  enough  that  they  should  be  added  to 
the  syllabus  to  become  part  of  standardized 
training;  and  2)  nonstandard  variations  of  ma¬ 
neuvers  and  tasks  which  are  used  frequently 
enough  that  they  should  be  added  to  the  menus 
of  standard  variations  available  to  the  in¬ 
structor  for  conducting  partially  specialized 
training. 
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Provisions  for  Different  Run  Types.  The  those  used  to  control  and  monitor  the  status 

proposed  console  design  also  makes  explicit  of  the  simulator  itself;  2)  those  used  to 

provisions  for  four  basic  types  of  runs  or  identify  and  modify  the  event  to  be  trained; 

"learning  events"  which  are  used  in  simulator  3)  those  used  to  monitor  and  control  the 

training:  IP  demonstration  runs;  automated  training  of  individual  maneuvers  and  tasks; 

demonstration  runs,  student  practice  runs;  and  4)  those  used  to  enter  numbers  into  the  com- 

replay  runs.  puter  (e.g.  to  change  the  value  of  parameters 

defining  initial  conditions,  etc.);  and  5) 

In  IP  demonstration  runs  the  instructor  those  used  to  insert,  monitor,  and  remove 

pilot  flies  the  simulator  from  the  instructor  malfunctions  in  the  simulated  aircraft.  The 

console  using  the  inside-out  and  outside-in  subpanels  containing  each  of  these  types  of 

graphic  displays  and  the  two-axis  controller.  switches  are  shown  below  in  Figures  3  and  4. 

This  feature  was  included  to  permit  the  in¬ 
structor  to  actively  fly  the  simulated  air-  The  Simulator  Status  Panel,  the  Numeric 

craft  in  order  to  illustrate  facets  of  perfor-  Entry  Panel,  and  the  Malfunction  Select  Panel 

mance  that  are  not  adequately  illustrated  by  are  self-explanatory.  The  functions  asso- 

any  of  the  automated  demonstrations.  Auto-  ciated  with  the  Event  Control  Panel  and  the 

mated  demonstration  runs  are  runs  flown  by  an  Maneuver  Control  Panel  are  described  below, 

interactive  autopilot  or  by  a  “canned"  tape. 

Student  practice  runs  (“live  runs")  and  replay  Event  Control  Panel .  The  Event  Control 

runs  are  self-explanatory.  The  use  of  the  in-  Panel  is  used  as  follows.  Pressing  the  STAGE 

structor  console  to  set  up  and  control  each  of  switch  causes  a  menu  of  the  different  stages 

these  types  of  runs  is  described  in  the  fol-  of  training  to  appear  on  the  CRT  display, 

lowing  sections.  e.g..  Instruments,  Formation,  ACM,  etc.  In¬ 

dicating  a  specific  stage  with  the  touch 

Controls.  Figure  2,  above  illustrates  the  screen  calls  up  a  menu  of  the  different  train- 
three  basic  types  of  controls  provided  on  the  ing  events  (simulator  sorties)  which  must  be 

instructor  control/display  module.  These  are:  flown  during  that  stage.  Indicating  a  spe¬ 

cific  event  on  this  menu  causes  the  detailed 

1.  A  transparent  CRT  touch  screen  used  to  lesson  plan  for  that  event  to  be  displayed, 

select  items  from  alphanumeric  or  pictorial  showing  the  sequence  of  maneuvers  and  tasks  to 

menus  displayed  on  the  CRT.  be  trained,  and  the  minimum,  standard,  and 

maximum  number  of  trials  for  each  maneuver  or 

2.  Dedicated  function  switches  located  task.  If  the  lesson  plan  is  more  than  one 

both  above  and  below  the  CRT.  These  are  page  long,  pressing  a  point  on  the  bottom 

grouped  into  five  subpanels  corresponding  to  right  (or  left)  corner  of  the  touch  screen 

five  basic  functions  performed  by  the  instruc-  will  cycle  forward  (or  backward)  through  the 

tor:  simulator  status  monitoring;  event  con-  pages  in  the  menu.  Indicating  a  specific 

trol;  maneuver  or  task  control;  number  entry;  event  also  causes  the  event  module  for  that 

and  malfunction  insertion.  The  switches  used  event  to  be  loaded  into  main  memory.  Pressing 

to  perform  each  of  these  functions  are  dis-  the  START  EVENT  switch  then  calls  up  the  task 

cussed  below.  module  for  the  first  maneuver  or  task  to  be 

trained  and  starts  setting  up  the  simulator 

3.  A  two-axis  controller  located  to  the  for  the  first  trial  of  the  first  maneuver  or 

right  of  the  control  switch  slant  panel.  (For  task.  (Where  desirable,  this  could  be  mech- 

use  with  an  IFT,  OFT  or  ACMT.  Not  required  anized  so  that  an  instructor  could  start  an 

for  a  CPT . )  The  two-axis  controller  can  be  event  from  somewhere  other  than  the  beginning 

used  by  the  instructor  a)  to  fly  the  student's  by  simply  touching  the  screen  to  indicate  the 

aircraft  during  manual  demonstrations  of  cor-  first  maneuver  or  task  to  be  trained  before 

rect  maneuver  performance  ("IP  demos");  b)  to  pressing  the  START  EVENT  switch.) 

manually  control  the  simulated  enenny  aircraft 

during  ACM  and  gunnery  training;  and  c)  to  fly  The  LESSON  PLAN  switch  shown  in  Figure  4 

one  of  the  friendly  aircraft  during  close  for-  can  be  used  to  call  up  the  lesson  plan  for  re- 

mation  or  traffic  pattern  training.  view  at  any  time  during  the  event.  The  lesson 

plan  can  also  be  used  as  a  menu  to  conduct 

Five  types  of  dedicated  function  indicator  specialized  training  by  introducing  maneuvers 

switches  were  defined  for  the  console:  1)  and  tasks  out  of  sequence,  e.g.,  to  skip  a 
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FIGURE  3.  SIMULATOR  STATUS  PANEL 
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section  of  the  plan,  or  to  return  to  a  maneu¬ 
ver  already  completed.  Similarly,  event  menus 
and  stage  menus  can  be  used  to  introduce  maneu¬ 
vers  normally  practiced  only  in  other  events 
or  even  maneuvers  normally  practiced  only  in 
other  stages.  Again,  the  more  radical  the 
departure  from  the  standardized  training  pre¬ 
specified  for  the  event,  the  greater  the  num¬ 
ber  of  procedural  steps  required. 

Maneuver  Control  Panel .  The  Maneuver  Con- 
trol  Panel  is  used  as  follows.  The  POSITION 
SIM,  START,  FREEZE,  and  STOP  switches  were 
carefully  selected  as  a  minimal,  necessary  and 
sufficient  set  of  simulator  run  controls  on 


the  basis  of  our  experience  in  conducting  sim¬ 
ulator  training  experiments  over  the  past  sev¬ 
eral  years.  The  function  of  these  switches  is 
described  below  in  Figure  9  which  presents  a 
detailed  sequential  description  of  instructor 
control  inputs  and  display  indications  for  a 
standard  prerecorded  maneuver  demonstration. 
These  controls  function  in  exactly  the  same 
way  for  an  instructor  flown  demonstration  (IP 
DEMO),  autopilot  or  prerecorded  demonstration 
(AUTO  DEMO),  student  practice  run  (LIVE  RUN), 
or  replay.  The  functions  of  the  NEXT  MANEUVER 
and  EXTRA  TRIAL  switches  are  described  below 
in  the  discussion  of  the  Quantitative  Perfor¬ 
mance  Display. 
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Displays.  Dynamic  "inside-out,"  and 
"outside-in"  pictorial  representations  of  the 
student's  aircraft  and  its  relation  to  other 
aircraft  or  ground  features  are  now  relatively 
common,  at  least  fo1"  certain  maneuvers.  For 
example,  many  current  instructor  consoles  can 
show  dynamic  two-dimensional  pictorial 
representations  of  aircraft  glideslope  and 
localizer  error.  Similarly,  dynamic  three- 
dimensional  perspective  outside-in  and  inside- 
out  pictorial  representations  of  ACM  engage¬ 
ment  like  those  developed  by  Cubic  for  the 
Tactical  Air  Combat  Training  System/Air  Combat 
Maneuvering  Range  (TACTS/ACMR)  have  been  de¬ 
veloped  for  ACM  simulator  instructor  consoles 
by  Northrop  and  other  companies.  (Spring, 
1976)  In  the  proposed  design,  dynamic  inside- 
out  and  outside-in  views  of  the  maneuver  being 
flown  are  combined  with  a  number  of  special 
purpose  displays  to  provide  the  instructor 
with  the  essential  information  required  at 
each  point  in  training.  Figure  5  shows  an 
outside-in  display  of  an  ACM  engagement.  The 
numbers  in  Figure  5  show  the  relative  position 
of  attacker  and  bogey  at  four  different  times 
during  the  engagement. 


As  noted  previously,  the  generalized  (all 
except  CRT)  configuration  of  the  instructor 
console  has  three  CRT  displays.  These  are: 

1.  A  small  CRT  at  the  top  of  the  left- 
hand  or  "secondary  display"  module.  This  is 
normally  used  as  a  repeater  display  to  show 
the  instructor  the  view  through  the  HUD  and 
the  HUD  symbology  being  seen  by  the  student. 

2.  A  21"  CRT  on  the  secondary  display 
module.  During  a  run  this  is  normally  used  to 
present  a  dynamic  outside-in  pictorial  rep¬ 
resentation  of  the  student's  aircraft  and  its 
relation  to  other  aircraft  or  features  on  the 
ground. 


3.  A  21"  CRT  on  the  interactive  control/ 
display  module.  During  a  run  this  is  normally 
used  to  present  a  dynamic  inside-out  pictorial 
representation  of  the  view  through  the  wind¬ 
screen  of  the  student's  aircraft.  Between 
runs  this  CRT  is  used  to  present  special- 
purpose  displays  which  provide  the  instructor 
with  a  variety  of  additional  information 
required  to  set  up,  conduct,  monitor,  and 
evaluate  training. 

A  description  of  the  primary  special- 
purpose  displays  developed  during  the  design 
study  is  presented  in  the  following  sections. 

Maneuver  Description  Display.  A  sample 
Maneuver  Description  Display  is  shown  in  Fig¬ 
ure  6.  This  display  is  automatically  pre¬ 
sented  to  the  instructor  at  the  beginning  of  a 
training  for  each  new  maneuver  or  task.  The 
display  shows: 

1.  The  type  and  number  of  demonstrations 
to  be  flow;  and  the  minimum,  standard  and  max¬ 
imum  number  of  student  practice  or  "live"  runs 
to  be  flown  following  the  demonstrations. 

2.  The  standard  or  "desired"  time  of  oc¬ 
currence  of  critical  points  or  "windows"  which 
normally  occur  during  the  maneuver,  on  a  time 
scale  appropriate  to  the  maneuver. 

3.  A  detailed  description  of  the  perfor¬ 
mance  standards,  initial  conditions,  and  per¬ 
formance  conditions  for  the  maneuver  to  be 

f 1  own . 

The  purpose  of  the  maneuver  description 
display  is  to  refresh  the  instructor's  memory 
as  to  a)  the  way  in  which  the  maneuver  should 
be  flown  and  b)  the  specific  criteria  which 
should  be  used  to  evaluate  student 
performance. 

Quantitative  Performance  Display.  A  typi¬ 
cal  Summary  Quantitative  Performance  Display 
is  shown  in  Figure  7.  This  display  is 
automatically  presented  to  the  instructor  at 
the  end  of  each  trial.  The  display  shows: 

1.  An  indication  of  the  training  status 
for  the  maneuver.  This  part  of  the  display  is 
unchanged  from  that  shown  in  the  Maneuver  De¬ 
scription  Display  except  that  the  number  of 
the  trial  just  completed  is  brightened  to  show 
the  instructor  the  number  of  trials  completed 
and  the  number  yet  to  be  trained. 

2.  Both  the  actual  and  desired  (standard) 
time  of  occurrence  of  each  critical  point  or 
"window"  for  the  maneuver.  This  display  was 
added  because  of  analytical  studies  which  sug¬ 
gest  that  the  times  of  occurrence  of  critical 
points  in  a  maneuver  are  sensitive  indicators 
of  overall  maneuver  performance. 

3.  A  description  of  the  most  important 
out-of-tolerance  conditions  detected  at  each 
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BARREL  ROLL  ATTACK  TRIALS  DEMO-1 
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MANEUVER:  BARREL  ROLL  ATTACK  FROM  ABEAM  PERCH 

PERFORMANCE  STANDARD:  END  UP  BEHIND  BOGEY  WITH : 

AOT  »  0  *  15° 

NOSE-TAIL  1500'  ♦  200  FT 
AIRSPEED  =  CO  SPEED  TO  +  15  KIAS 
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AOT  =  90° 

NOSE-TAIL  =  0 

SLANT  RANGE  =  1.6  Ml  PERFORMANCE  CONDITIONS 

VERTICAL  SEPARATION  -  +2000  FT  FUEL  =  2000  LBS 

AIRSPEEO  =  350  KIAS  BOGEY  AIRSPEED  =  350  KIAS 


FIGURE  6.  MANEUVER  DESCRIPTION  DISPLAY 


window  or  in  the  segment  or  “corridor"  immedi¬ 
ately  following  that  window.  The  out-of- 
|  tolerance  conditions  for  windows  and  corridors 

.  are  detected  by  the  automated  performance  mon¬ 
itoring  system  discussed  in  the  following 

;  section. 

As  illustrated  in  Figure  7,  the  Summary 
Quantitative  Performance  Display  shows  out-of- 
j  tolerance  conditions  detected  for  each  window 

|  and  succeeding  segment.  Each  out-of-tolerance 

condition  is  indicated  by  a  very  brief  English- 
'  language  description  of  the  out-of -tolerance 

i  condition,  followed  by  the  observed  value  of 

I  the  parameter  and  the  standard  or  desired 

i  value  in  parentheses.  For  example.  Figure  7 

1  shows  that  the  pull-up  occurred  very  late, 

,  that  it  was  initiated  when  the  attacker  was 

•  1/2  miles  from  the  bogey,  and  that  is  should 

|  have  been  initiated  when  the  attacker  was  ap- 

|  proximately  7/8  of  a  mile  from  the  bogey.  The 

;  values  in  parentheses  serve  to  continually  re¬ 

mind  the  Instructor  of  the  qualitative  perfor¬ 
mance  standards  established  for  the  maneuver. 

It  would  of  course  be  possible  to  present 
this  same  display  to  the  student  by  projecting 
It  on  the  simulator's  image  presentation  sys¬ 
tem  or  by  display  1i>y  it  on  the  student's  mul¬ 
tifunction  display  in  the  cockpit.  This 
feature  could  be  used  as  a  tool  to  assist  the 
Instructor  In  providing  feedback  to  the  stu¬ 
dent.  With  appropriate  controls  added  to  the 
cockpit  this  could  also  be  used  to  provide  the 


simulator  with  a  capability  for  self- 

instruction,  which  can  be  highly  efficient 

when  alternated  with  conventional  instruction.  • 

It  should  be  noted  that  the  display  pre¬ 
sents  only  the  two  or  three  most  important 
out-of-tolerance  conditions  detected  for  each 
window  and  accompanying  segment.  This  is  done 
to  prevent  overloading  the  instructor  with  too 
much  information.  It  is  done  by  means  of  a  ♦  1 

table  look-up  system  which  identifies  the  most 
important  errors  by  consulting  a  table  con-  _ 

taining  the  prespecified  rank  order  of  impor-  ' 

tance  of  each  of  the  known  errors  for  each 
window  and  segment  for  the  maneuver.  This 
table  is  part  of  the  automated  performance  ] 

monitoring  algorithm  which  is  in  turn  part  of  • 

the  task  module  for  the  maneuver.  If  de-  ►  - 

sired,  the  instructor  can  obtain  information  , 

on  additional  performance  information  for  any  o 

window  and  segment  by  simply  touching  the  name  J 

of  the  window.  This  causes  a  Detailed  Quanti¬ 
tative  Performance  Display  of  all  the  out-of¬ 
tolerance  conditions  detected  for  that  window  m 

and  segment  to  be  displayed.  This  detailed  -  • 

display  for  a  given  window  and  segment  would 
probably  group  out-of- tolerance  conditions  by 
parameter  type  to  make  It  easier  for  the  in¬ 
structor  to  read,  e.g.,  under  headings  such  as 
“Energy  Parameters,"  and  "Relative  Geometry 
Parameters."  m 

The  time  scale  at  the  top  of  the  display 
has  a  second,  possibly  very  useful  function. 
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FIGURE  7.  SUMMARY  QUANTITATIVE  PERFORMANCE  DISPLAY 


By  pressing  any  two  points  on  the  scale  the 
instructor  can  preset  the  starting  and  stop¬ 
ping  points  of  any  automated  maneuver  demon¬ 
stration  or  replay.  This  might  be  done  to 
save  time  or  to  highlight  some  aspect  of  ma¬ 
neuver  performance.  Thus,  in  the  example 
shown  in  Figure  7,  the  instructor  could  use 
this  technique  to  quickly  set  up  the  simulator 
to  replay  a  just  completed  live  run  from  a 
point  just  before  the  low  reversal  to  a  point 
just  after  the  inverted  position,  in  order  to 
illustrate  some  detail  of  correct  or  incorrect 
performance  in  that  part  of  the  maneuver. 


A  similar  capability  can  be  provided  for 
live  runs  by  developing  additional  software  to 
enable  the  computer  to  pick  off  nonstandard 
initial  conditions,  etc.  from  intermediate 
points  in  a  canned  demonstration  of  the  ma¬ 
neuver.  The  ability  to  set  up  arbitrary  start 
and  stop  points  for  live  runs  provides  an  in¬ 
herent  capability  for  backward  chaining  and 
similar  instructional  techniques  as  discussed 
below  in  the  section  on  instructional 
features. 

In  addition  to  enabling  the  instructor  to 
keep  track  of  where  he  is  in  the  training  for 
each  maneuver  the  part  of  the  display  showing 
the  trials  completed  and  trials  remaining  are 
used  by  the  instructor  in  connection  with  the 
NEXT  MANEUVER  and  EXTRA  TRIAL  switches  shown 
above  in  Figure  4. 


Normally,  as  soon  as  the  standard  number 
of  student  practice  trials  (“live  runs")  have 
been  completed  for  a  given  maneuver,  the  in¬ 
structor  will  debrief  the  student  and  then 
press  the  NEXT  MANEUVER  switch.  This  will 
cause  the  simulator  computer  to  call  up  the 
task  module  for  the  next  maneuver  specified  in 
the  event  module  and  start  setting  up  the 
simulator  for  the  first  trial  of  the  next 
maneuver,  typically  a  demonstration.  If  the 
standard  number  of  practice  runs  have  not  been 
completed  but  the  instructor  decides  that  no 
further  practice  is  necessary,  and  if  the  num¬ 
ber  of  trials  completed  is  equal  to  or  greater 
than  the  minimum  shown  on  the  display,  the  in¬ 
structor  can  move  directly  to  the  next  maneu¬ 
ver  to  by  pressing  the  NEXT  MANEUVER  switch. 
Similarly,  if  the  standard  number  of  practice 
runs  have  been  completed  but  the  instructor 
decides  the  student  needs  more  practice,  and 
if  the  number  of  trials  flown  is  less  than  the 
maximum  shown  on  the  display,  the  instructor 
can  set  up  an  extra  trial  on  the  same  maneuver 
simply  by  pressing  the  EXTRA  TRIAL  switch.  In 
this  way  the  instructor  can  easily  vary  the 
number  of  trials  flown  within  prespecified 
limits  to  tailor  training  to  the  student’s 
needs.  He  cannot  exceed  these  limits  without 
going  through  a  more  complex  procedure.  This 
feature  is  designed  to  make  it  difficult  to 
reduce  the  number  of  trials  to  the  extent  that 
a  student  receives  little  or  no  practice  on  a 
maneuver,  or  to  increase  the  number  of  trials 
to  the  extent  that  insufficient  time  is  left 
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for  training  on  remaining  maneuvers.  Here 
again,  the  actual  number  of  trials  flown  on 
each  maneuver  could  be  recorded  to  provide  an 
empirical  basis  for  modifying  the  minimum, 
standard,  and  maximum  number  of  trials  spec¬ 
ified  In  the  event  module. 

Other  Displays.  Several  other  displays 
were  developed  to  reduce  instructor  workload 
for  various  tasks.  For  example  two- 
dimensional  tabular  menus  were  developed  to 
enable  the  instructor  to  quickly  set  up  the 
desired  mode  of  controlling  each  aircraft  in  a 
three  ship  formation,  or  in  two-on-one  or  one- 
on-two  ACM  engagements.  This  type  of  display 
is  illustrated  in  Figure  8. 
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FIGURE  8.  MENU  DISPLAY  FOR  SELECTING  METHOD 
OF  CONTROLLING  EACH  AIRCRAFT  FOR  ACM 


Detailed  Functions  Analyses.  Detailed 
functions  analyses  have  been  developed  showing 
exactly  how  the  Event  Control  Panel ,  Maneuver 
Control  Panel,  Maneuver  Description  Display 
and  Summary  Quantitative  Performance  display 
would  be  used  to  carry  out  each  of  the  follow¬ 
ing  instructional  functions  and  subfunctions: 

Identify  event  to  be  trained 

Conduct  standardized  training 

-  demonstrate  maneuver 

-  instruct  live  run 

-  replay  live  run 

Conduct  partially  standardized  training 

-  modify  start-stop  points 

-  modify  number  of  trials  per  maneuver 

•  normal  transition 

•  skip  to  next  maneuver 

t  give  extra  trials  on  current 

maneuver 

-  introduce  standard  variation  of 
maneuver 

Conduct  fully  specialized  training 

-  introduce  nonstandard  variation  of 
maneuver 

-  introduce  maneuver  out-of-sequence 

-  introduce  maneuver  not  contained  in 
event  module 


Insert  and  remove  malfunction 

Specify  method  of  controlling  each  aircraft 
for  ACM  and  Formation  Flights. 


A  sample  functions  analysis  for  demonstrat¬ 
ing  a  maneuver  is  shown  in  Figure  9.  This 
type  of  analysis  is  an  important  first  step  in 
developing  instructor  console  software  in  that 
it  can  fairly  easily  be  translated  into  a  com¬ 
puter  flow  diagram. 


Automated  Performance  Monitoring 


System 


As  discussed  in  the  previous  section,  the 
automated  performance  monitoring  (APM)  system 
generates  a  quantitative  performance  display 
at  the  end  of  a  maneuver  which  is  a  summary  of 
the  most  important  out-of-tolerance  conditions 
detected  during  the  maneuver.  An  example  of  a 
quantitative  performance  display  is  shown 
above  in  Figure  7.  This  information  is  pre¬ 
sented  to  the  instructor  as  an  aid  for  evalu¬ 
ating  and  diagnosing  student  performance  on 
the  maneuver. 


The  APM  system  is  based  primarily  on  tech¬ 
niques  developed  by  Northrop,  Vought,  and  Sys¬ 
tems  Technology,  Inc.  (Carter  1976;  Carter 
1977;  Sepp  1977;  Heffley  et.  al.  1982)  It 
also  uses  APM  concepts  developed  by  Appli- 
Mation,  Inc.,  the  Canyon  Research  group,  and 
Vreuls  Research,  Inc.  (Semple,  et.  al.  1979; 
Vreuls,  et.  al .  1975) 


The  system  is  designed  to  detect  three 
types  of  errors:  flight  path  errors,  control 
technique  errors,  and  procedural  (switch  se¬ 
quence)  errors.  In  all  cases,  errors  or  out- 
of-tolerance  conditions  are  detected  by 
comparing  the  observed  value  of  a  parameter 
for  a  specific  point  or  segment  in  a  maneuver 
o>  task  with  the  standard  value  and  tolerance 
limits  stored  in  computer  memory.  Flight  path 
errors,  e.g.,  "excessive  airspeed  at  roll  in," 
are  currently  being  measured  in  this  way  by 
the  Vought  A-7  NCLT  APM  system  now  in  opera¬ 
tional  use  at  NAS  Lemoore.  Control  technique 
errors,  e.g.,  "controlling  attitude  instead  of 
Nz"  can  be  measured  in  this  way  by  pilot  be¬ 
havior  identification  techniques  like  those 
developed  by  STI,  which  were  used  in  Northrop 
simulation  studies  to  evaluate  the  effective¬ 
ness  of  alternate  training  aircraft  configur¬ 
ations  for  carrier  landing.  Procedural 
errors,  e.g.,  "arming  switch  thrown  out  of 
sequence"  are  now  being  detected  in  this  man¬ 
ner  by  the  EA-3  Low-Cost  CPT  developed  by 
Appli-Mation,  Inc.,  and  several  other  systems. 

The  potential  impact  of  APM  techniques  on 
the  effectiveness  and  efficiency  of  simulator 
training  has  been  demonstrated  by  studies  on 
the  inter-  and  intra-observer  reliability  of 
instructor  judgments.  (Carter,  1976;  Knoop  & 
Welde,  1973)  These  studies  show  that  instruc¬ 
tor  judgments  of  overall  maneuver  performance 
are  fairly  reliable,  but  that  instructor  judg- 
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CONDUCT  STANDARDIZED  TRAINING 


Following  the  identification  of  the  event  to  be  trained,  the  instructor  conducts  a  standardized  training  event  by  using  the  Maneuver 
Control  Panel  and  the  special  purpose  displays  as  follows: 

1.  Demonstrates  Maneuver  (AUTO  DEMO) 

a.  Presses  "POSITION  SIM"  switch  which  causes  the  "POSITION  SIM  '  light  to  start  blinking  and  the  computer  to  start  setting 
up  the  simulator  for  the  prerecorded  demonstration  specified  for  this  maneuver  m  the  task  module. 

b.  ("POSITION  SIM"  switch  comes  on  steady  and  first  frame  of  HUD,  flight  path,  and  instrument  repeater  dynamic  displays 
are  displayed  on  instructor  console  CRTs,  indicating  that  simulator  is  ready  to  fly.) 

c.  Comments  on  the  demonstration  to  be  flown  and  verifies  that  the  student  is  ready.  P 

d.  Presses  "START"  switch  to  start  the  prerecorded  demonstration. 

e.  Makes  comments  during  the  demonstration. 

f.  Presses  "FREEZE"  switch  to  freeze  demonstration  in  mid-air.  (This  freezes  the  visual  scene,  motion  platform,  g-seat, 
g-suit,  instruments,  etc,  in  the  position  existing  at  time  of  freeze.) 

g.  Presses  "START"  switch  to  restart  prerecorded  demonstration  from  frozen  position. 

h.  Instructor  presses  "STOP"  switch/demonstration  reaches  auto  stop  point. 

(1)  Turns  off  visual  scene  and  returns  motion  platform,  g-seat,  g-suit,  instruments,  etc,  to  a  neutral  position. 

(2)  Freezes  last  frame  of  HUD  and  flight  path  displays.  P 

(3)  Causes  Summary  Quantitative  Performance  Display  and  Grading  Scale  Display  to  appear  on  the  interactive  CRT. 

FIGURE  9.  SAMPLE  FUNCTIONS  ANALYSIS 


ments  relating  to  the  acceptability  of  spe¬ 
cific  parameters  at  specific  points  or  over 
specific  segments  during  the  maneuver  are 
highly  unreliable.  Since  these  judgments  are 
the  basis  for  the  instructor's  diagnosis  of 
the  causes  of  substandard  maneuver  perfor¬ 
mance,  it  follows  that  much  of  the  feedback 
currently  given  to  students  is  in  error.  The 
effect  of  this  on  the  effectiveness  and  effi¬ 
ciency  of  training  can  be  understood  by  con¬ 
sidering  the  effect  of  telling  a  student  he 
pulled  up  late  when  he  actually  pulled  up 
early,  which  is  precisely  what  the  data  indi¬ 
cate  is  happening. 

The  proposed  APM  system  is  designed  to 
correct  this  problem  by  providing  instructors 
with  objective  data  which  can  be  used  to  give 
students  more  accurate  feedback.  This  same 
data  can  be  recorded  to  provide  an  objective 
data  base  for  improved  management  of  student 
progress,  improved  student  selection,  and  more 
accurate  evaluation  of  total  system 
performance. 

Although  the  benefits  of  the  proposed  APM 
system  are  extensive,  the  cost  of  developing 
and  perfecting  an  APM  algorithm  for  a  single 
complex  maneuver  can  be  quite  high.  For 
large-scale  training  systems  a  phased  APM  de¬ 
velopment  may  be  required  to  spread  these 
costs  over  time.  In  this  approach  algorithms 
are  first  developed  only  for  selected,  skill- 
sensitive  maneuvers.  Algorithms  for  addi¬ 
tional  maneuvers  can  then  be  developed  as  part 
of  an  ongoing  ISO  effort. 

INSTRUCTIONAL  FEATURES 

A  review  of  current  literature  on  instruc¬ 
tional  features  served  as  a  starting  point  for 
the  identification  and  evaluation  of  candidate 
instructional  features,  (e.g.,  Bailey  & 
Hughes,  1980;  Caro  et.  al.  1979,  1980;  Hughes, 
1978,  1979;  Lintern  &  Gopher,  1977;  Lintern 
1978;  Semple,  et.  al.  1980;  Shaw  1979;  and 
Weller,  1979)  Candidate  instructional 


features  were  classified  into  two  broad  types: 
nonautomated  and  automated. 

Nonautomated  Features.  Figure  10  illus- 
trates  the  results  of  an  analysis  performed  to 
evaluate  a  variety  of  nonautomated  instruc¬ 
tional  features  identified  as  candidates  for 
simulator  training.  In  general,  the  features 
shown  as  rejected  were  rejected  because  of  a 
total  lack  of  empirical  data  on  their  training 
effectiveness. 

Backward  chaining  has  been  found  to  be  a 
highly  effective  simulator  training  technique 
in  an  experimental  setting.  (Hughes,  1979) 

It  is  therefore  recommended  for  IFT,  OFT,  and 
ACMT  training  pending  further  investigations 
of  its  cost-effectiveness  in  military  pilot 
training  programs.  The  instructor  console  fea¬ 
ture  used  to  set  up  arbitrary  start  and  stop 
points  for  live  runs  provides  a  basic  capabil¬ 
ity  for  backward  chaining  and  similar  instruc¬ 
tional  techniques,  as  discussed  above  in  the 
section  on  instructor  console  displays. 

Fast  time  has  been  found  to  be  very  effec¬ 
tive  simulator  training  technique  at  NASA-Ames 
for  preparing  NASA  test  pilots  for  the  severe 
time-compression  effect  encountered  in  actual 
test  flights.  Fast  time  is  a  technique 
whereby  the  program  integration  interval  is 
greater  than  the  elaosed  real-time  interval, 
causing  frequencies  and  velocities  to  in¬ 
crease,  i.e.,  things  happen  more  quickly  than 
in  real  flight.  Fast  time  is  thus  a  form  of 
overtraining  which  could  conceivably  increase 
the  training  effectiveness  of  IFT  and  OFT 
training  by  better  preparing  students  for  the 
increased  stress  and  workload  associated  with 
aircraft  flights.  Like  other  types  of  over¬ 
training  it  could  also  result  in  excessive 
simulator  training  time.  While  current  simu¬ 
lator  computers  have  an  inherent  capability 
for  fast  time,  the  cost  of  upgrading  image 
generation  computers,  motion  system  servos, 
and  other  hardware  to  achieve  the  required 
frequency  response  would  be  very  high.  This 
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FEATURE 

REJECTED 

SELECTED 

1  PARAMETER  FREEZE  .CAPABII  ITY  FOR 

FREEZING  ONE  OR  MORE  SELECTED  PARA 

X 

METERS.  EG  RANGE  TO  TARGET) 

2  BACKWARD  CHAINING  (CAPABILITY  FOR 

IFT,  OFT  AND  ACMT 

TRAINING  LAST  SEGMENT  IN  A  MANEUVER  FIRST, 
PROGRESSIVE!  Y  ADDING  EARLIER  SEGMENTSl 

(SEE  DISCUSSION) 

3  FAST  TIME  (CAPABILITY  FOR  FLYING  SIMU¬ 
LATOR  AT  EASTER  THAN  REAL  TIMEl 

X 

4  SLOW  TIME  (CAPABILITY  f  OR  FLYING  SIMU 
LATOR  AT  SL.OWE  R  THAN  REAL  TIME) 

X 

5  NON  INTERACTIVE  DYNAMIC  VISUAL  SCENE 

FOR  CPT  (DISPLAY  ENABLING  STUDENT  TO 

CORRELATE  PROCEDURES  WITH  THE  SIGHT 
PICTURE.  E  G  ,  FOR  LEARNING  TRAFFIC 

PATTERN  PROCEDURE) 

X 

6  INSTRUMENT  BLANKING  (CAPABILITY  FOR 
BLANKING  OUT  SELECTED  INSTRUMENTS  FOR 
SCAN  TRAINING) 

X 

7  EYE-TRACK  DISPLAY  (DYNAMIC  DISPLAY 
SHOWING  INSTRUCTOR  WHERE  STUDENT  IS 
LOOKING  AT  EACH  MOMENT) 

X 

8  MANUAL  REPOSITIONING  (CAPABILITY  FOR 
INSTRUCTOR  TO  CHANGE  ATTITUTE  AND  ALTI¬ 
TUDE  OF  STUDENTS  AIRCRAFT  DURING  FREEZE 

TO  ILLUSTRATE  CHANGE  IN  THE  SIGHT  PICTURE 
(A  TYPE  OF  PARAMETER  FREEZEl 

X 

9  AUGMENTED  VISUAL  CUES 'PREDICTOR  DIS¬ 
PLAYS  TO  BE  DISPLAYED  ON  HUD 

OFT,  ACMT 

FIGURE  10.  EVALUATION  OF  NON-AUTOMATED  INSTRUCTIONAL  FEATURES 


technique  is  therefore  not  recommended  pending 
a  more  direct  demonstration  of  its  effective¬ 
ness  for  military  flight  training. 

Landing-point  predictor  displays  and  aug¬ 
mented  visual  cues  such  as  the  "pole  track" 
display  are  recommended  as  selectable  OFT  HUD 
displays  for  field  and  carrier  landing  train¬ 
ing.  The  "pole  track"  is  a  display  of  succes¬ 
sively  shorter  "poles"  on  each  side  of  the 
glideslope  showing  the  correct  guideslope  and 
providing  an  external  indication  of  aircraft 
velocity.  The  effectiveness  of  such  displays 
for  simulator  training  in  field  and  carrier 
landings  has  been  demonstrated  in  the  studies 
by  Lintern  (1978)  and  Weller  (1979). 

Electronic  tracer  bullets  and  funnel  dis¬ 
plays  like  those  on  the  G.E.  HUD  are  recom¬ 
mended  as  selectable  ACMT  HUD  displays  for 
air-to-air  gunnery  training.  These  displays 
have  been  shown  to  be  extremely  effective  for 


teaching  a  student  the  effect  of  aircraft 
control  inputs  on  the  dynamics  of  the  bullet 
stream.  A  continuously  computed  impact  point, 
velocity  vector,  and  bomb  fall  line  are  recom¬ 
mended  as  selectable  OFT  HUD  displays  for  air- 
to-ground  weapon  delivery  training.  It  is 
assumed  that  these  HUD  displays  would  dupli¬ 
cate  those  in  the  aircraft. 

Automated  Features.  Figure  11  illustrates 
the  results  of  an  analysis  performed  to  evalu¬ 
ate  automated  instructional  features  which 
were  identified  as  candidates  for  an  ACMT. 


The  capability  for  preprogrammed  initial 
conditions  and  preprogrammed  malfunctions  is 
inherent  in  the  task  module  concept.  Tech¬ 
niques  for  using  normal  and  emergency  proce¬ 
dure  task  modules  in  conjunction  with  other 
types  of  task  modules  are  described  in  Semple, 
et.  al.  (1980). 
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FEATURE 

— 

REJECTED 

SELECTED 

PREPROGRAMMED  INITIAL  CONDITIONS 

CPT,  1  FT,  OFT,  ACMT 
(TASK  MODULE) 

PREPROGRAMMED  MALFUNCTIONS 

CPT,  IFT,  OFT,  ACMT 
(TASK  MODULE) 

AUTOMATED  GROUND  CONTROLLER 

IFT.  OFT 

PRERECORDED  BOGIES  AND  FRIENDLY  A/C 

OFT.  ACMT 

MANEUVER-SPECIFIC  AUTOPILOT  BOGIES  AND 
FRIENDLY  AIRCRAFT 

OFT,  ACMT 

GENERALIZED  AUTOPILOT  BOGEY 

ACMT (ADVANCED 
TRAINING  ONLY) 

AUTOMATED  PERFORMANCE  MONITORING 

CPT.  IFT,  OFT.  ACMT 

AUTOMATED  AUDIO  ALERTS  DURING  MANEUVER 
(TO  INSTRUCTOR) 

CPT,  IFT,  OFT,  ACMT 

AUTOMATED  GRADING 

OFT  ONLY 

(AT  THE  INSTRUCTORS  CONSOLE) 

(RECOMMENDED  FOR 
FIELD  AND  CARRIER 
LANDING) 

AUTOMATED  FEEDBACK  AFTER  MANEUVER 
(TO  STUDENT) 

CPT  ONLY 

AUTOMATED  INSTRUCTION 

CPT  ONLY 

ADAPTIVE  TRAINING 

OFT  ONLY 

(USE  OF  HUD  SYMBOLOGY) 

AUTOMATED  COACHING  DURING  MANEUVER 
(TO  STUDENT) 

X 

FIGURE  11.  EVALUATION  OF  AUTOMATED  INSTRUCTIONAL  FEATURES 


An  automated  ground  controller  similar  to 
that  developed  for  the  F-14  OFT  Instructional 
Support  System  is  recommended  for  IFTs  and 
OFTs  for  simulating  the  ground  controller's 
voice  instructions  in  ground-controlled  ap¬ 
proaches  and  carrier-controlled  approaches. 
Instructor  pilots  consulted  agreed  that  in¬ 
structor  pilots  cannot  accurately  simulate 
ground  controller  voice  instructions  due  to 
the  specialized  training  and  practice  re¬ 
quired.  This  feature  should  thus  increase  the 
training  effectiveness  of  IFTs  and  OFTs  while 
reducing  instructor  workload. 

Maneuver-specific  "autopilot  bogies"  are 
recommended  for  controlling  the  simulated 
bogey  and  wing-man  in  Introductory  ACM  train¬ 
ing  and  for  controlling  the  lead  aircraft, 
second  aircraft,  or  third  aircraft  in  close- 
and  tactical -formation  training.  This  ap¬ 
proach  was  used  with  success  in  ACM  simulator 
training  experiments  performed  by  Northrop  for 
NADC  In  1976.  (Spring,  1976;  Carter,  1976) 

The  autopilot  bogey  Is  used  for  introductory 
training  In  classic  maneuvers.  Typically,  It 
Is  a  relatively  simple  algorithm  which  causes 
the  bogey  or  friendly  aircraft  to  react  to  a 


change  in  student  aircraft  position  in  the 
same  way  that  the  instructor  would  react  at 
this  stage  of  training.  For  example,  a  barrel 
roll  attack  bogey  is  programmed  to  tighten  its 
turn  as  the  student  reduces  angle-off-: tail  and 
nose-to-tail ,  and  to  reverse  when  the  student 
overshoots.  Autopilot  bogies  increase  OFT  and 
ACMT  training  effectiveness  in  two  ways:  1) 
they  provide  much  more  realistic  training  than 
is  possible  with  noninteractive  prerecorded 
bogies;  and  2)  they  significantly  reduce  the 
need  for  the  simulator  instructor  to  fly  the 
bogey  aircraft  in  ACM  and  formation  training, 
thus  permitting  the  instructor  to  concentrate 
on  observing  and  coaching  student  performance. 

The  term  "generalized  autopilot  bogey"  is 
used  here  to  refer  to  an  autopilot  bogey  which 
is  capable  of  flying  an  entire  ACM  engagement 
against  a  human  pilot.  This  type  of  bogey  is 
used  in  the  Northrop  Large  Amplitude  Simulator 
(LAS),  the  Simulator  for  Alr-to-Air  Combat 
(SAAC)  at  Luke  AFB,  and  the  2E6  ACM  simulator 
at  NAS  Oceana.  A  generalized  autopilot  bogey 
is  recommended  for  ACMTs  to  be  used  in  ad¬ 
vanced  training  but  would  probably  not  be 
cost-effective  for  undergraduate  ACM  training. 
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As  shown  in  Figure  11,  an  automated  per¬ 
formance  monitoring  system  is  recommended  for 
all  four  types  of  simulators.  As  described  in 
the  previous  section,  the  APM  system  detects 
out-of-tolerance  conditions  during  each  trial 
of  each  maneuver  or  task,  and  displays  this 
information  to  the  instructor  at  the  end  of 
the  trial.  By  providing  more  detailed  and 
more  accurate  data  on  student  performance,  the 
APM  system  performs  three  essential  functions. 
First,  it  increases  the  effectiveness  of  ac¬ 
tive  instruction  by  providing  the  instructor 
with  objective  information  which  can  be  used 
as  an  aid  in  evaluating  and  diagnosing  indi¬ 
vidual  trials.  Second,  it  puts  the  management 
of  student  progress  through  the  syllabus  on  a 
more  objective  basis  by  providing  instructors 
with  more  complete  and  accurate  data  on  which 
to  base  grades.  Finally,  it  provides  a 
greatly  increased  system  evaluation  capability 
by  providing  operational  ISO  personnel  with 
objective  data  on  the  effects  of  variations  in 
training  content,  sequence,  techniques,  and 
equipment. 

Automated  audio  alerts  are  recommended  for 
all  four  types  of  simulators  to  alert  the  in¬ 
structor  to  certain  critical  student  errors 
which  might  otherwise  be  missed.  This  enables 
the  instructor  to  correct  errors  as  soon  as 
they  occur  to  prevent  formation  of  unsafe 
habits,  such  as  pulling  too  many  Gs.  This  is 
done  by  means  of  audio  tones  and/or  digitally 
recorded  voice  messages. 

Automated  verbal  coaching  and  cueing  of 
the  student  during  a  maneuver  is  not  recom¬ 
mended  because  of  research  evidence  that  such 
coaching  is  at  best  distracting,  and  at  worst 
constitutes  an  irrelevant  cue  to  correct  ma¬ 
neuver  performance  which  can  be  used  con¬ 
sciously  or  unconsciously  to  "beat  the 
system." 

Automated  grading  is  recommended  for  field 
and  carrier  landing  training  in  OFTs  because 
of  the  demonstrated  agreement  between  A-7  NCLT 
Landing  Signal  Officer  (LSO)  grades  and  the 
grades  assigned  by  the  A-7  NCLT  automated 
grading  system  developed  by  Vought. 

A  capability  for  fully  automated  Computer- 
Assisted  Instruction  (CAI)  is  recommended  only 
for  CPTs.  At  the  completion  of  each  procedure 
this  system  provides  automatic  feedback  to  the 
student  by  displaying  the  errors  and  the  cor¬ 
rect  procedures  on  the  instructor/student  con¬ 
sole  CRT  display.  The  system  then  instructs 
the  student  to  reset  all  switches  to  their 
normal  positions.  Depending  on  performance  to 
that  point,  it  then  instructs  the  student  to 
perform  the  previous  procedure  again  or  to 
start  training  on  a  new  procedure. 

Adaptive  training  is  recommended  for  OFTs 
in  conjunction  with  an  A-7  NCLT-type  grading 
system  to  vary  the  difficulty  of  field  or  car¬ 
rier  landing  as  a  function  of  student  perfor¬ 


mance.  Task  difficulty  is  decreased  by 
displaying  a  predictor  display  and  augmented 
visual  cues  such  as  a  "pole  track"  display  on 
the  student’s  HUD.  Task  difficulty  is  in¬ 
creased  by  removing  these  displays.  While  it 
is  possible  for  the  instructor  to  insert  and 
delete  these  displays  manually,  the  training 
effectiveness  of  these  displays  is  greatest  if 
they  are  inserted  and  deleted  according  to  an 
optimized  adaptive  training  algorithm. 
(Lintern,  1978) 

Adaptive  training  in  which  task  difficulty 
is  varied  by  varying  system  dynamics,  (e.g., 
changing  the  stability  of  the  simulated  air¬ 
craft)  has  been  shown  to  be  ineffective  and  is 
not  recommended.  (Shaw,  1979). 


BRIEF/DEBRIEF  CONSOLE 

The  brief/debrief  console  for  the  proposed 
ISS  is  essentially  a  CAI  terminal  interfaced 
to  a  videodisc  system,  which  has  an  added 
video  cassette  playback  capability  for  de¬ 
briefing.  It  is  assumed  that  the  brief/ 
debrief  console  would  be  interfaced  to  a  com¬ 
puterized  scheduling  and  records  keeping 
system. 

Physical  Configuration 

The  physical  configuration  of  the  console 
consists  of  a  21"  CRT  display  with  a  transpar¬ 
ent  membrane-type  touch  screen  and  a  set  of 
dedicated  function  switches  located  on  an  off¬ 
screen  membrane-type  touch  panel,  where  indi¬ 
vidual  switches  are  located  and  grouped  by 
means  of  cutouts  in  a  thick  plastic  overlay. 
Ideally,  the  brief/debrief  console  would  be 
identical  to  the  CAI/videodisc  terminals  used 
for  academic  training,  with  the  exception  of 
the  video  cassette  unit  and  additional  func¬ 
tion  switches  used  only  for  debriefing.  A 
detachable  alphanumeric  keyboard  is  required 
to  permit  the  IP  to  enter  nonstandard  comments 
in  the  students  computer  file. 

Functional  Design 

The  console  is  designed  to  be  used  for 
both  simulator  and  aircraft  flights.  It  sup¬ 
ports  five  major  instructional  functions,  as 
fol lows: 

1.  Event  Planning.  The  console  is  used 
by  instructors  to  determine  student  status;  to 
review  the  student's  performance  in  past 
events;  to  request  schedule  changes  if  neces¬ 
sary;  and  to  identify  minor  changes  in  the 
standard  lesson  plan  which  could  benefit  the 
student. 

2.  CAI  Briefing  and  Test.  The  console  is 
used  to  give  students  a  CAI  review  and  test  on 
academic  and  flight  support  materials  relevant 
to  the  flight  to  be  flown. 
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3.  IP  Briefing.  The  console  can  provide 
a  hard  copy  of  the  CAI  test  results  to  be  used 
by  the  IP  as  a  starting  point  for  the  IP  brief' 
Ing  which  follows  the  CAI  briefing.  (A  hard 
copy  printer  could  be  time  shared  by  several 
brief/debrief  consoles.) 

4.  Debrief ing.  The  console  can  be  used 
for  debriefing  both  simulator  and  aircraft 
flights,  assuming  that  the  aircraft  has  an 
onboard  video  recorder, 

5.  Data  Entry.  The  console  is  used  by 
the  IP  to  enter  grades  and  comments  on  the 
flight  in  the  student's  computer  file. 

Debriefing  Displays 

For  simulator  debriefings,  a  multiplexing 
scan  converter  is  used  to  make  a  two-channel 
videotape  record  of  what  was  shown  during  and 
after  each  maneuver  on  any  two  of  the  instruc¬ 
tor  console  CRT  displays.  Thus  one  videotape 
channel  will  typically  show  a  dynamic  "inside- 
out"  presentation  of  the  pilot's  view  through 
the  windscreen  during  the  maneuver,  including 
continuously  changing  digital  readouts  of  key 
parameters  on  the  edges  of  the  display.  This 
is  followed  by  the  Summary  Quantitative  Per¬ 
formance  Display  shown  to  the  instructor  at 
the  end  of  the  maneuver,  plus  any  detailed 
quantitative  performance  displays  requested  at 
that  time.  The  second  channel  will  typically 
show  a  dynamic  "outside-in"  presentation  of  an 
outside  observer's  view  of  the  maneuver,  also 
including  digital  readouts.  This  is  followed 
by  a  static  graphic  display  of  the  desired  and 
actual  flight  path  flown.  The  instructor  can 
switch  back  and  forth  between  these  displays 
as  required  during  debriefing. 

Debrief  Procedure 

For  debriefing,  the  brief/debrief  console 
is  used  by  the  IP  and  the  student  to  review 
the  videotape  record  of  the  aircraft  or  simu¬ 
lator  flight  to  be  debriefed.  For  both  air¬ 
craft  and  simulator  debriefings  it  is  expected 
that  only  selected  segments  of  the  tape  would 
be  reviewed,  since  reviewing  the  entire  tape 
would  require  as  much  time  as  the  original 
flight.  To  make  this  process  as  efficient  as 
possible,  three  different  methods  are  provided 
to  enable  the  IP  to  quickly  get  to  a  point  of 
interest  on  the  tape.  First,  he  can  slew  di¬ 
rectly  to  any  electronic  flags  which  he  in¬ 
serted  on  the  tape  during  the  event.  Second, 
he  can  slew  to  a  specific  trial  of  a  specific 
maneuver,  using  a  display  of  the  lesson  plan 
as  a  menu.  (This  approach  will  be  most  effec¬ 
tive  for  simulator  debriefings,  due  to  fre¬ 
quent  changes  in  lesson  plans  which  are 
dictated  by  events  and  conditions  in  the  air.) 
Third,  he  can  use  fast-forward  and  rewind  con¬ 
trols  to  slew  to  an  approximate  time  in  the 
flight,  where  the  approximate  flight  time  at 
each  point  on  the  tape  will  be  continuously 


displayed  on  the  screen.  Once  the  approximate 
point  Is  located  by  one  of  these  three  meth¬ 
ods,  the  precise  segment  of  the  tape  to  be  re¬ 
played  will  be  controlled  by  conventional  tape 
playback  controls  (fast-forward,  rewind,  stop, 
pause,  and  play.)  Detailed  functions  analyses 
have  been  developed  which  describe  the  proce¬ 
dures  used  for  slewing  to  a  desired  point  on 
the  tape  with  each  method. 


Data  Entry  Procedure 

The  instructor  grades  each  event  using  the 
console  touch  screen  and  an  event-specific 
grading  form  displayed  on  the  CRT.  The  grad¬ 
ing  form  consists  of  a  matrix  of  grading  cat¬ 
egories  listed  vertically  down  the  left  side 
of  the  display  and  a  grading  scale  displayed 
horizontally  across  the  top.  To  grade  a  spe¬ 
cific  category,  the  instructor  simply  presses 
the  touch  screen  across  from  the  category  in 
the  column  corresponding  to  the  desired  grade. 
Standard  comments  on  the  students  performance 
are  entered  by  pressing  the  touch  screen  next 
to  one  or  more  appropriate  comments  in  a  menu 
of  standard  comments.  Nonstandard  comments 
are  typed  in  with  the  alphanumeric  keyboard. 

It  is  assumed  that  all  grades  and  comments 
entered  by  the  instructor  at  the  console  would 
be  automatically  entered  into  a  centralized 
computer  file  containing  a  record  of  the  stu¬ 
dent's  performance  in  each  event. 


CURRENT  STATUS  AND  FUTURE  PLANS 

An  independent  research  and  development 
project  is  now  in  progress  to  develop  a  work¬ 
ing  model  of  the  Summary  Quantitative  Perfor¬ 
mance  Display.  Current  plans  are  to  follow  up 
this  effort  by  developing  working  models  of 
the  other  instructor  console  features  described 
above. 
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INTRODUCTION 

The  Offensive  Avionics  System  (OAS)  PPT 
(See  Figure  2)  Includes  two  user  Interfaces, 
one  for  the  crewmembers  and  one  for  the 
Instructor.  The  Instructor  can  Interact  with 
the  CPT  In  three  operational  nodes:  Pre-Run, 
Run,  and  Post-Run.  The  Interface  has  been 
designed  to  facilitate  moving  back  and  forth 
anong  any  of  the  three  nodes  (See  Figure  1). 
The  Pre-Run  node  occurs  prior  to  the  training 
(simulation)  session  and  allows  the 
Instructor  to  select  and  edit  a  scenario  for 
training  and  complete  setup  and  Initial¬ 
ization  procedures.  The  Rim  mode  is  the 
actual  training  period  during  which  the 
Instructor  monitors  the  crewnenbers'  actions 
and  Injects  changes  into  the  session.  The 
Post-Run  node  occurs  after  the  training 
session  In  which  the  Instructor  can  Initiate 
a  limited  review  and  analysis.  All  sessions 
nust  go  through  some  minimum  initialization 
via  the  Pre-Run  mode  of  the  user  Interface. 
Depending  on  the  desired  degree  of 
interaction.  Initialization  can  vary  widely. 
After  the  appropriate  Initialization,  the 
instructor  will  enter  the  Run  mode  to  begin 
the  training  session.  He  will  be  provided 
with  commands  that  allow  him  to  abort  the 
session  and  return  to  the  Pre-Run  mode  or 
Interrupt  for  a  time  period  and  then  resume 
within  the  Run  mode  itself.  Once  the  run  Is 
complete,  the  instructor  enters  the  Post-Run 
mode  In  which  he  may  conduct  limited  review 
and  analysis  or  he  may  return  to  the  Pre-Run 
node  to  initiate  another  training  session. 


Separate  areas  or  windows  will  be  set  aside 
for  various  types  of  Information. 

The  largest  area  at  the  top  of  the 
console  Is  reserved  for  two  major  functions. 
In  the  Pre-Run  mode  this  area  Is  used  for 
displaying  tables  of  values  describing  flight 
path  characteristics  of  a  selected  scenario. 
During  the  Run  mode,  this  area  shall  be 
dedicated  to  the  real-time  display  and  update 
of  a  set  of  variables  denoted  as  real-time 
parameters.  During  Post-Run  mode,  this  sres 
•hall  also  be  used  to  reflect  reel-time 
persmeter  updstes,  but  shell  reference  only  a 
subset  of  those  parameters  used  during  the 
Run  mode. 

The  Commend  Menu  area,  at  the  bottom 
left,  will  be  used  In  sll  three  nodes  to 
dlsplsy  the  commends.  In  menu  form,  evalleble 
to  the  instructor  for  selection.  As  the 
Instructor  transits  from  one  menu  to  another, 
this  area  will  display  the  new  menu. 

The  area  located  to  the  right  and  top  of 
the  menu  consists  of  two  lines  displaying 
faults  end  pilot-controlled  ststus 
informstlon.  This  sres  will  be  active  only 
during  Run  mode  and  will  be  empty  during  both 
Pre-Run  and  Post-Run  modes.  The  first  line 
of  the  sres,  the  Fsults  Injected  line,  will 
lndlcste  sny  mslf unctions  or  faults  currently 
In  effect.  The  second  line  shall  display  the 
current  status  of  both  the  Pilot  Steering 
mode  (msnusl/euto)  and  Terrain  Avoidance  mode 
(on/off). 


Figure  1 

Instructor  Interface 


COMMANDS  AND  DISPLAY  LAYOUT 

Within  any  given  mode,  the  Instructor  has 
a  set  of  commends  availsble  for  carrying  out 
his  responsibilities.  The  command  menus  snd 
other  information  will  be  presented  on  sn 
alphanumeric  CRT  laid  out  as  in  Figure  3. 


In  sll  three  nodes,  prompts  to  the 
Instructor  for  Input  and  notification  of 
errors  will  appear  In  tha  Prompts  and  Error 
Messages  area.  If  the  Instructor 
Inadvertently  enters  any  form  of  unacceptable 
Input,  he  will  be  notified,  told  the  nature 
of  the  mistake,  and  requested  to  reenter  the 
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desired  Input.  The  Instructor  Input  line  Is 
active  In  sll  three  modes  and  Is  the  only 
line  able  to  receive  and  echo  Input  from  the 
Instructor.  The  bottom  line  of  the  terminal 
Is  hardwsre-controlled  and  outputs  terminal 
status  Information. 

The  tables  which  can  be  viewed  by  the 
Instructor  during  the  Pre-Run  mode  are  used 
for  purposes  of  preview,  editing,  and 
scenario  generation.  The  Information 
contained  In  the  tables  describe  the  selected 
flight  plan  values  and  cover  such  Items  as 
destination  points,  air  speed,  missile 
Information,  etc.  These  entries  may  be 
altered  by  the  Instructor  to  develop  a 
different  flight  plan  and  can  even  be 


by  typing  the  number  of  the  desired  command. 
All  keyboards  Input  will  be  displayed  In  the 
instructor  Input  area  of  the  screen.  Once 
the  number  has  been  Input,  the  Instructor 
merely  has  to  hit  the  CARRIAGE  RETURN  (CR) 
key  to  activate  the  selected  command.  If  the 
command  then  requires  additional  Input  such 
as  a  numeric  value  before  activation  can  be 
completed,  the  Instructor  will  be  given  a 
prompt  Indicating  what  information  Is 
required  and  what  will  have  to  be  typed  in. 
Editing  keys  will  be  provided  to  allow  the 
Instructor  to  recover  from  typographical 
errors.  Once  the  numeric  value  desired  Is 
displayed  In  the  active  field,  the  Instructor 
again  hits  the  CARRIAGE  RETURN  for  final 
input  into  the  system.  This  menu-select- 


PRE-RUN: 

SCENARIO 

RUN: 

REAL-TIME  PARAMETERS 

1  FAULTS  INJECTED 

1  PILOT  STEERING  AND  TA  MODE  STATUS 
l__  .  _ 

COMMAND  MENU 

1 

1  PROMPTS  AND  ERROR  MESSAGES 

1 

l . 

1  INSTRUCTOR  INPUT 

TERMINAL  STATUS 

Figure  3 

Instructor  Console  Layout 


"trimmed”  down  to  create  a  much  shorter 
scenario. 

A  wide  variety  of  variables  are  displayed 
during  the  Run  mode.  These  variables  relate 
such  Information  as  aircraft  status, 
navigational  system  accuracy,  INS  drift, 
crewmember  Input,  system  state  values,  etc. 
Once  displayed,  the  values  of  these  variables 
are  continuously  updated  at  S-second 
intervals. 

A  menu  of  available  commands  will  be 
presented  in  the  Command  Menu  area.  The 
command  labels  will  be  numbered  on  the 
display  and  may  be  selected  by  the  Instructor 


to  -activate  design  Is  aimed  at  minimizing 
the  amount  of  typing  required  by  the 
instructor.  It  also  has  the  advantages  of 
always  displaying  the  available  commands, 
eliminating  any  errors  due  to  command-typing 
mistakes,  and  allowing  the  user,  via  the 
numerical  select  procedure,  to  quickly  access 
and  activate  commands  of  interest. 

SYSTEM  EXECUTIVE 

When  the  mainframe  is  brought  up  and 
Initialized,  the  first  thing  that  the 
instructor  sees  is  the  EXECUTIVE  MEN  (See 
Figure  4)  which  consists  of  the  three  simple 
mode  selections  of  Pre-Run,  Run,  and  Post 
Run.  In  the  hierarchy  of  menu  organization, 
the  instructor  can  always  return  to  the 
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EXECUTIVE  menu  to  change  inodes  any  time 
during  a  training  run.  From  this  menu,  the 
Instructor  may  choose  to  enter  any  of  the 
three  modes  or  exit  the  training  program 
entirely.  In  the  event  the  instructor 
activates  the  Run  or  Post-Run  modes  without 
having  specified  a  working  scenario,  a 
default  value  will  be  assumed  and  scenario  #1 
will  be  loaded  into  the  system  as  the  current 
working  scenario.  Once  the  instructor 
selects  any  of  the  modes,  the  terminal 
displays  the  master  menu  for  that  mode. 

Using  the  commands  in  these  master  menus,  the 
Instructor  can  move  to  other  subsets  of 
commands  that  allow  him  to  carry  out  any  of 
the  tasks  required  in  the  selected  mode.  At 
any  time,  the  instructor  can  activate  the 
RETURN  TO  EXECUTIVE  command  available  in  each 
of  the  master  menus  and  return  to  EXECUTIVE 
menu  to  select  a  new  mode. 


EXECUTIVE  MENU 

1)  PRE-RUN 

2)  RUN 

3)  POST-RUN 

4)  EXIT 


Figure  4 

The  Executive  Menu 


OPERATIONAL  MODES  AND  FORMAT  OF  COMMANDS 

The  instructor  will  be  provided  with  a 
set  of  seven  pre-canned  scenarios  and 
material  describing  each  of  these,  including 
navigational  charts.  The  seven  scenarios 
will  be  permanently  numbered  1  through  7  and 
will  be  available  at  any  time  for  the 
instructor's  use.  There  is  additional  space 
for  three  more  scenarios,  numbered  8  through 
10,  which  can  be  generated  by  the  instructor 
through  modification  of  any  of  the  available 
scenarios  and  stored  in  these  last  three 
slots.  Thus,  there  are  a  total  of  ten 
possible  scenarios,  seven  permanent  ones 
(numbers  1  through  7)  and  three  replaceable 
ones  (numbers  8  through  10). 

Upon  entering  the  Pre-Run  mode,  one  of 
the  first  requirements  of  the  instructor  is 
to  select  a  scenario  by  recalling  one  of  the 
ten  that  are  stored.  Once  recalled,  all  of 
the  Initialization  values  stored  under  that 
scenario's  number  will  be  loaded  into  the 
mainframe.  This  scenario  now  becomes  the 
"working"  scenario  and  can  be  used  for  the 
training  session  as  is  or  edited  to  create  a 
new,  modified  working  scenario.  Note  that  if 
a  working  scenario  is  not  explicitly 


selected,  the  default  working  scenario  will 
be  scenario  #1. 

The  canned  scenarios  can  be  modified  by 
the  Instructor.  Modifications  to  the  working 
scenario  are  divided  into  two  categories: 

(1)  direct  alteration  of  specific  flight  leg 
characteristics  (such  as  aircraft  speed, 
altitude,  wind  velocity,  etc.),  and  (2) 
scenario  reduction.  The  first  editing 
capability  is  accomplished  by  identifying 
specific  flight  legs  of  the  working  scenario, 
and  replacing  current  flight  plan  values  with 
different  values.  Scenario  reduction  is 
achieved  by  specifying  a  contiguous  set  of 
flight  legs  that  exist  in  the  working 
scenario.  This  set  becomes  the  new  "reduced" 
scenario,  and  retains  all  flight 
characteristics  as  defined  for  the 
corresponding  legs  of  the  original  working 
scenario.  All  editing  functions  which  create 
a  new  scenario  description  cause  the  original 
working  file  to  be  replaced  by  the  new 
scenario  which  then  becomes  the  current 
working  scenario.  Only  the  working  scenario 
stored  in  the  mainframe  is  changed  by  the 
editing  functions;  the  scenario  stored  on  the 
disk  remains  unchanged.  When  the  instructor 
has  completed  editing  a  scenario,  he  may 
choose  to  save  a  copy  of  the  working  scenario 
for  future  retrieval.  If  the  copy  is  not 
saved,  all  modification  and  initialization 
which  has  occurred  during  Pre-Run  will  be 
lost  when  the  CPT  program  is  exited.  If  no 
editing  has  occurred,  it  is  not  necessary  to 
save  the  working  scenario  since  it  is  merely 
a  copy  of  one  of  the  scenarios  already 
stored. 

PRE-RUN  COMMANDS 

The  four  commands  which  are  available  for 
selection  during  Pre-Run  mode  are  listed  in 
menu  form  in  Figure  5.  When  activated,  each 
command  will  either  produce  a  new  menu  of 
sub-commands,  or  will  cause  a  request  for 
additional  Information  in  the  form  of  a 
prompt.  In  the  following  sections  each 
command  will  be  addressed  in  detail  with  the 
corresponding  display  changes  and  required 
inputs  clearly  outlined. 


PRE-RUN  MENU 
1}  SELECT  SCENARIO 

2)  EDIT/PREVIEW  SCENARIO 

3)  SAVE  SCENARIO 

- 4)  RETURN  TO  EXECUTIVE 

I 


Figure  5 
Pre-Run  Menu 
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Activation  of  the  SELECT  SCENARIO  command 
is  required  before  any  manipulation  of  any 
scenario  other  than  scenario  #1  may  be 
accomplished.  Uhen  the  SELECT  SCENARIO 
command  has  been  activated  (by  having  the 
Instructor  type  1  CR  ),  the  command  menu 
will  remain  on  the  screen,  but  a  prompt  will 
appear  requesting  a  scenario  file  number. 

The  Instructor  must  then  input  a  number  from 
1  to  10  (representing  the  ten  scenario  files) 
followed  by  a  CR  .  This  will  define  the 
scenario  file  which  will  become  the  current 
working  scenario.  Any  previous  working 
scenario  will  be  replaced  by  this  selection. 
Should  the  Instructor  decide  not  to  define  a 
(new)  working  scenario,  he  may  enter  a  CR  by 
itself.  This  will  be  accepted  as  a  negation 
to  the  original  command  and  control  will 
resume  at  the  command  menu  level .  Any 
illegal  input  will  be  flagged  as  such  in  the 
error  and  prompt  area  of  the  console  and  the 
instructor  will  be  requested  for  new  input. 

By  entering  2  CR  ,  the  Instructor 
activates  the  EDIT/PREVIEW  SCENARIO  command 
which  causes  a  new  command  menu  to  be 
displayed  (see  Figure  6).  Each  command  in 
the  menu  is  related  to  the  display  of 
particular  data  tables  or  values  providing 
Information  describing  certain  attributes  of 
the  working  scenario.  These  tables  or  values 
can  be  requested  for  purposes  of  preview  by 
the  Instructor,  or  can  be  edited  for 
production  of  a  new  scenario. 


EDIT/PPEVIEW  SCENARIO 

1 

1)  FLIGHT  PLAN 

i 

2)  SRAM  TARGET  TABLE 

1 

3)  DESTINATION  TABLE 

1 

4)  FIX-POINT  TABLE 

1 

5)  MISSILE  STATUS 

i 

- 6)  RETURN  TO  PRE-RUN 

1 

1 

1 

Figure  6 

Edit/Preview  Scenario  Menu 


The  first  command  on  the  menu  is  the 
FLIGHT  PLAN  command  (see  Figure  6).  Entering 
a  1  CR  will  cause  the  first  page  (11  lines) 
of  the  Flight  Plan  table  to  appear  in  the 
scenario  display  area  (upper  window),  and  the 
FLIGHT  PLAN  menu  to  appear  in  the  Command 
Menu  area.  The  Flight  Plan  table  lists  the 
destination  points  and  flight  characteristics 
associated  with  each  flight  leg  in  the 
working  scenario.  The  FLIGHT  PLAN  menu 
contains  a  set  of  commands  allowing  editing 


and  scrolling  of  the  flight  plan.  The 
editing  commands  will  explicitly  call  out 
only  those  columns  in  the  flight  plan  which 
may  be  edited.  Whenever  the  instructor 
decides  to  change  one  entry  in  the  table,  all 
other  entries  which  are  affected  by  that 
change  will  be  automatically  replaced  and 
updated  on  the  display. 

The  second  command  available  in  the 
EDIT/PREVIEW  SCENARIO  menu  is  SRAM  TARGET 
TABLE  (see  Figure  6).  Activated  by  a  2  CR 
keystroke  input,  the  SRAM  Target  Table  is 
displayed  in  the  scenario  display  area,  and 
the  SRAM  TARGET  TABLE  menu  appears  in  the 
command  menu  area.  All  targets  in  the 
scenario  are  listed  in  the  table  by  target 
number,  with  the  flight  leg  destination 
number  and  target  characteristics  such  as 
latitude,  longitude,  and  elevation  displayed 
in  the  corresponding  row.  The  only  two 
columns  which  are  available  for  editing  by 
the  instructor  are  SAIR  (Safe  And  In  Range) 
ENTRY  and  SAIR  EXIT. 

The  Destination  Table  (see  Figure  6)  is 
called  up  by  selecting  3  CR  from  the  EDIT/ 
PREVIEW  SCENARIO  menu  and  presents  the 
working  scenario  flight  plan  by  listing  each 
destination  point,  type  of  destination  point, 
latitude,  longitude,  elevation,  and  planned 
time  of  arrival.  The  DESTINATION  TABLE  menu 
will  also  be  displayed  in  the  command  menu 
area.  No  editing  of  specific  values  in  the 
Destination  Table  is  possible,  but  the 
Instructor  may  create  a  new  scenario  by 
reducing  the  current  working  scenario  by 
selecting  a  contiguous  sub-portion  of  it. 

This  sub-portion  will  then  become  the  new 
working  scenario. 

The  Fix-Point  Table  (4  CR)  is  only 
available  for  presentation;  no  editing  may 
occur.  This  table  will  list  all  fix  points 
which  may  be  accessed  within  the  limits  of 
the  working  scenario.  The  points  will  be 
listed  sequentially  by  number  with  the 
corresponding  latitude,  longitude,  and 
elevation  of  each  point  displayed  as  well. 
This  table  will  usually  exceed  display 
capacity  in  size  and  will  require  a  paging 
capability. 

By  entering  a  5  CR  ,  the  Instructor  can 
change  Missile  Status.  This  parameter  is  set 
to  either  1  (Fully  Aligned)  or  2  (Power  Off) 
in  the  scenario.  The  Fully  Aligned  status 
will  cause  the  elimination  of  the  required 
warm-up  time  of  missiles  and  weapons  within  a 
scenario.  It  will  initialize  the  scenario 
with  missiles  fully  targeted  (powered,  armed, 
and  in  "GO"  status)  and  MIUs  fully  powered. 
The  Power  Off  status  assumes  no  warm-up  of 
missiles  or  weapons  has  occurred.  Upon 
entering  the  command,  no  change  in  the  menu 
occurs  and  nothing  is  displayed  in  the 
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scenario  area.  Rather,  a  message  appears  in 
the  prompt  and  error  area  Indicating  the 
current  value  of  the  missile  status  parameter 
and  requesting  a  new  value.  Entering 
anything  other  than  1  CR  or  2  CR  will 
cause  an  error  message  to  appear  and  a 
request  for  new  input.  Entering  CR  only 
will  negate  the  command  giving  control  to  the 
EDIT/PREVIEW  SCENARIO  menu. 

When  the  Instructor  selects  the  RETURN  TO 
PRE-RUN  (6  CR),  the  EDIT/PREVIEW  SCENARIO 
command  menu  will  clear  from  the  display  area 
and  control  will  return  to  the  PRE-RUN  master 
command  menu  (see  Figure  6). 

If  the  instructor  wishes  to  save  the 
current  working  scenario  and  its 
corresponding  parameters  for  later 
retrieval/usage,  he  must  activate  the  SAVE 
SCENARIO  command  by  entering  3  CR  in  the 
PRE-RUN  master  menu  (See  Figure  5).  A  prompt 
will  appear  requesting  a  scenario  file  number 
for  8  to  10  (representing  the  three 
replaceable  files  available  for  storage). 

Once  selected,  the  old  contents  of  the  file 
will  be  destroyed  and  replaced  by  the  current 
working  scenario.  If  the  instructor  decides 


not  to  save  the  working  scenario,  he  may 
enter  a  CR  by  itself.  This  will  negate  the 
command  activation,  and  control  will  return 
to  the  PRE-RUN  master  command  menu.  All 
illegal  inputs  will  be  flagged  as  such  to  the 
instructor  and  he  will  be  requested  for  new 
input . 

Activation  of  this  command  will  clear  the 
PRE-RUN  master  command  menu  from  the  display 
and  control  will  return  to  the  EXECUTIVE 
command  menu  (See  Figure  5). 

RUN  MODE 

The  instructor  activates  the  Run  mode  by 
entering  2  CR  from  the  EXECUTIVE  menu. 

Upon  entering  this  command,  the  training 
session  (simulation)  starts. 

Throughout  the  training  session,  a  set  of 
real-time  parameters  reflecting  aircraft, 
navigational,  and  command  information  are 
displayed  in  the  upper  window  of  the  console 
and  updated  every  5  sec  (see  Figure  7).  The 
simulated  time  is  displayed  at  the  top, 
right-side.  Below  that  Is  the  aircraft  data 


r  I 

AIRCRAFT  DATAj 

TATI — FT47-00.00  TH:  305 
LONG:  W1 19-00.00  TK:  300 

[TIME  1 4 : 1 2 : 30f 

ALT:  21500  DEST:  16  B28-TGT 

W/V:  300/50  TAS:  420 

fiAVIGATIONAL  DATAI 

LAT:  N  47-15.10  TH:  300 
LONG:  W120-10.00  TK:  310 

ALT:  22100  PTA:  L  00:16:13 

W/V:  310/30 

Cast  command  issuer 

RN-IK3:  XHAIR  1 0,1 2 

N-IKB:  MDFY  24 

RNMP:  RANGE/SCALE  75 

PANEL:  WCP  —  LP  1,2,3  WPN  PWR 

RUN  MENU 

1)  FAULT  INJECTION 

2)  AIRCRAFT  MANEUVER 

3)  TAL  RECOGNIZED 

4)  TERRAIN  AVOIDANCE  MODE 

5)  FREEZE 

6)  RESUME 

7)  RETURN  TO  EXECUTIVE 

I  RNMP-FAIL  WCP-FAIL 

1  PILOT  STEERING:  MANUAL  TA  MODE:  OFF 

1 _ 

1 

1  (PROMPTS  AND  ERROR  MESSAGES) 

1 

| . . . 

f  (INSTRUCTOR  INPUT) 

(TERMINAL  STATUS) 

consisting  of:  latitude  and  longitude,  true 
heading  and  ground  track,  altitude  and 
wlnd/veloclty ,  current  destination  point,  and 
true  air  speed.  Next  Is  the  navigational 
data  reflecting  values  for  the  prime 
navigational  system.  These  parameters  are: 
latitude  and  longitude,  true  heading  and 
ground  track,  altitude  and  wind/velocity,  and 
the  planned  time  of  arrival  error  (early  or 
late).  Navigational  errors  can  be  determined 
from  these  parameters  by  subtracting 
corresponding  values  of  the  aircraft  and 
navigational  data. 

The  bottom  portion  of  the  window  displays 
the  last  command  Issued  by  the  crewmembers  — 
separately  for  the  three  major  panels: 

RN-IKB,  N-IKB,  and  RWiP.  The  last  command 
Issued  for  the  remaining  set  of  panels  Is 
designated  by  a  panel  label  followed  by  the 
command  given. 

This  display  of  real-time  parameters 
allows  the  instructor  to  have  immediate, 
completely  updated  Information  as  to  the 
aircraft  state  and  crewmember  actions. 

In  addition  to  the  real-time  parameter 
display,  entering  the  Run  mode  causes  the  RUN 
menu  to  be  displayed  In  the  command  menu 
area.  There  are  seven  commands  associated 
with  the  Run  menu  and  the  following  sections 
will  explain  each  of  these  in  detail. 

RUN  MODE  COMMANDS 

The  Instructor  has  the  capacity  to  Inject 
malfunctions,  or  faults,  at  any  time  during 
Run  mode  (see  Figure  7).  This  allows  him  to 
cause  failures  to  occur  at  critical  times 
corresponding  with  crewmember  activity.  The 
fault  specified  will  occur  at  the  moment  of 
selection.  He  also  has  the  ability  to 
terminate  a  given  malfunction  that  Is 
currently  in  progress. 

Upon  entering  a  1  CR  ,  the  FAULT 
INJECTION  menu  will  appear  In  the  command 
menu  area  (see  Figure  8).  The  menu  provides 
four  fault  commands  and  a  RETURN  TO  RUN 
command . 


FAULT 

INJECTION  ! 

1) 

RN  MANAGEMENT  PANEL  i 

2) 

WEAPONS  CONTROL  PANEL 

3) 

DOPPLER  RADAR  , 

4) 

OAS  BUS  FAILURE 

i -  5) 

RETURN  TO  RUN 

Figure  8 

Fault  Injection  Menu 


Selection  of  the  RN  Management  Panel 
command  (1  CR  )  will  cause  the  current 
status  of  the  RN  Management  Panel  (RIMP)  to 
be  Indicated  In  the  prompt  and  error  area 
[1 (FUNCTIONING),  2  (FAILED)]  along  with  a 
request  for  Input.  If  the  current  status  Is 
FUNCTIONING  and  the  Instructor  enters  a  2 
CR,  the  R1WP  will  Immediately  cease  to 
function,  and  the  message  RMtP-FAIL  will 
appear  on  the  Faults  Injected  line  of  the 
Instructor's  console.  If  the  current  status 
Is  FAILED  and  a  1  CR  is  entered,  the  RNMP 
will  begin  to  function  again  and  the 
RNMP-PAIL  message  will  be  erased.  If  a  CR 
Is  Input  by  Itself,  the  command  will  negate 
and  the  fault  status  will  remain  unchanged. 

An  Input  of  any  value  other  than  1  or  2  will 
cause  an  error  message  and  a  request  for  new 
Input. 

Validation  checking  will  occur  by  the 
system  prior  to  activating  any  fault.  Since 
no  other  fault  can  occur  simultaneously  with 
an  OAS  bus  failure,  the  system  will  check  the 
OAS  bus  status  before  Initiating  other 
failures.  If  the  OAS  bus  status  Is  FAILED 
and  the  Instructor  attempts  to  Inject  an  RNMP 
failure,  an  error  message  will  appear  In  the 
prompt  and  error  area  and  RNMP  status  will 
remain  FUNCTIONING. 

Weapon  Control  Panel  failure  (see  Figure 
8)  is  activated  by  entering  2  CR  .  It 
operates  exactly  as  the  RN  MANAGQ1ENT  PANEL 
command  except  that  the  message  WCP-FAIL 
appears  In  the  Faults  Injected  area  of  the 
console. 

The  Doppler  Radar  failure  (see  Figure  8) 
Is  activated  by  entering  3  CR  .  It 
operates  exactly  as  the  RN  MANAGEMENT  PANEL 
command  except  that  the  message  DOPPLER-FAIL 
appears  in  the  Faults  Injected  area  of  the 
console. 

The  OAS  Bus  Failure  (see  Figure  8) 
differs  from  the  other  fault  commands  in  that 
once  this  fault  Is  injected.  It  must  remain 
FAILED  for  the  rest  of  the  training  session. 
Thus,  there  is  no  need  to  prompt  for  an  Input 
value  for  the  command. 

When  4  CR  Is  entered,  the  system  will 
check  the  status  of  the  other  three  panels. 

If  any  of  them  are  currently  FAILED,  an  error 
message  will  appear  and  the  command  will  be 
ignored.  If  all  three  of  the  other  panels 
are  currently  FUNCTIONING,  the  system  then 
checks  the  OAS  Bus  status.  If  It  Is 
currently  FAILED,  another  error  message  will 
appear  and  the  command  will  be  Ignored. 
Finally,  If  all  three  panels  and  the  OAS  Bus 
status  are  currently  FUNCTIONING  when  the 
command  Is  activated,  the  system  changes  the 
OAS  bus  status  to  FAILED  and  the  message  OAS 
BUS-FAIL  appears  In  the  Faults  Injected  area 
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of  the  console.  Once  the  Instructor  has 
Injected  this  failure,  he  will  have  no 
further  need  to  return  to  the  Fault  Injection 
menu  throughout  the  remainder  of  the  training 
session. 

Entering  5  CR  (see  Figure  8)  will  clear 
the  FAULT  INJECTION  menu  from  the  display 
area  and  control  will  return  to  the  RUN  menu 
(see  Figure  7). 

During  the  progression  of  the  training 
session,  the  instructor  has  the  capability  of 
acting  as  a  simulated  pilot  in  taking  the 
aircraft  through  changes  in  speed,  altitude, 
heading,  and  steering  mode.  He  can  also 
change  the  wind/velocity  (direction  and 
magnitude)  and  the  alternate  navigational 
heading  error. 

Upon  entering  2  CR  ,  the  AIRCRAFT 
MANEUVER  menu  will  be  displayed  in  the 
command  menu  area  (see  Figure  9).  This  menu 
consists  of  seven  commands  for  maneuvering 
the  aircraft,  charging  conditions,  or 
returning  to  the  RUN  menu. 


AIRCRAFT  MANEUVER 

1)  TAS 

2)  ALTITUDE 

3)  HEADING 

4)  PILOT  STEERING  MODE 

5)  W/V 

6)  ALTER  NAV  HEADING  ERROR 

7)  RETURN  TO  RUN 


Figure  9 

Aircraft  Maneuver  Menu 


When  TAS  is  selected  by  the  instructor 
with  a  (1  CR),  a  prompt  will  be  issued  for 
input  of  a  new  air  speed  value.  The  current 
air  speed  value  is  displayed  continually  and 
updated  in  the  real-time  parameter  display 
window. 

Once  a  new  value  has  been  successfully 
input,  the  aircraft  will  begin  a  constant 
acceleration/deceleration  until  it  reaches 
the  desired  velocity.  The  real-time 
parameter  will  reflect  this  change. 

As  with  the  aircraft  velocity,  when  the 
ALTITUDE  command  (see  Figure  9)  is  activated 
by  entering  2  CR  ,  a  prompt  will  be  issued 
for  input  of  a  new  altitude.  The  current 
altitude  is  another  real-time  parameter  that 
is  displayed  in  the  upper  window. 

Once  a  new  altitude  has  been  successfully 


input,  the  aircraft  will  begin  a  constant 
2000  ft/mtn  climb  or  descent  until  it  reaches 
the  desired  altitude.  The  real-time  ALT 
parameter  will  reflect  the  updated  altitude. 

After  a  new  altitude  has  been  input  by 
the  instructor,  the  aircraft  remains  at  that 
new  altitude  until  further  input  from  the 
instructor. 

The  instructor  has  the  capability  to 
input  a  new  heading  by  selecting  a  3  CR 
command  (see  figure  9).  Before  changing  the 
heading,  the  system  will  check  the  Pilot 
Steering  Mode.  In  order  for  the  pilot 
(instructor)  to  manually  change  the  heading, 
the  Pilot  Steering  Mode  must  be  MANUAL.  If 
the  Pilot  Steering  Mode  is  AUTO,  no  change  in 
heading  can  be  initiated  by  the  instructor, 
an  error  message  will  appear,  and  the  command 
will  be  ignored.  If  the  Pilot  Steering  Mode 
is  MANUAL,  the  HEADING  Command  will  be 
initiated. 

After  successfully  entering  a  new 
heading,  the  aircraft  will  begin  a  turn  of 
constant  radius  until  the  new  heading  is 
reached.  Once  attained,  if  this  heading  does 
not  take  the  aircraft  to  the  expected 
destination  point,  it  will  remain  in  effect 
until  a  new  heading  value  is  Issued. 

The  Pilot  Steering  Mode  can  be  changed 
(see  Figure  9)  by  entering  4  CR  .  The 
current  value  of  the  mode  [l(AUTO),  2 
(MANUAL) 1  will  be  displayed  in  the  prompt  and 
error  area  along  with  a  prompt  to  enter  a  new 
value.  Entering  a  value  other  than  1  or  2 
will  cause  an  error  message  to  appear  and  a 
prompt  for  a  new  input.  The  current  setting 
of  the  Pilot  Steering  Mode  is  also  displayed 
on  the  console  below  the  Faults  Injected 
line.  At  the  start  of  the  training  session, 
the  Pilot  Steering  Mode  will  default  to  AUTO. 

Entering  5  CR  allows  the  instructor  to 
update  the  wind/velocity  (direction  and 
magnitude).  A  prompt  for  a  new  value  of 
wind/velocity  will  appear.  The  new  value  of 
W/V  entered  by  the  instructor  will  remain  in 
effect  until  the  next  turn  point  in  the 
scenario  is  reached.  After  the  turn  point 
the  next  wind  will  be  obtained  from  the 
(scenario)  flight  plan.  All  changes  to 
wind/velocity  will  be  reflected  in  the 
real-time  W/V  parameter  value. 

The  instructor  can  change  the  Alternate 
Navigation  Heading  Error  (see  Figure  9)  by 
entering  6  CR  .  A  prompt  for  a  new  value, 
not  to  exceed  +5,  will  appear.  Any  illegal 
input  by  the  instructor  will  be  flagged  with 
an  error  message  and  a  prompt  for  a  new  input 
will  appear. 

If  the  Alternate  Navigation  system  is  the 
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prime  navigational  model,  the  instructor  can 
determine  the  current  alternate  navigation 
heading  error  by  subtracting  the  real-time  TH 
values  for  aircraft  and  navigational  data. 
Upon  entering  a  new  acceptable  value  for  the 
alternate  navigation  heading  error,  the 
alternate  navigation  system  will  replace  the 
old  error  with  this  new  value  and  update  the 
TH  parameter  to  reflect  the  new  error.  Any 
other  navigation  parameters  affected  by  the 
change  (such  as  TK)  will  also  be  updated. 

If  the  alternate  navigation  system  is  not 
the  prime  navigational  model,  the  instructor 
cannot  determine  the  current  alternate 
navigation  heading  error.  He  can  still  enter 
a  value  within  the  +5  limit  and  this  new 
error  will  be  stored  in  the  alternate 
navigation  system  replacing  the  old  value. 
However,  the  aircraft  and  the  current  prime 
navigational  model  will  not  act  upon  this  new 
value  until  the  alternate  navigation  system 
is  selected  as  the  prime  navigational  model. 

Entering  7  CR  will  replace  the  AIRCRAFT 
MANEUVER  menu  with  the  RUN  menu  (see  Figure 
7)  and  pass  control  on  to  it. 

When  missiles  and  weapons  are  powering  up 
and  reach  the  Transfer  Alignment  (TAL)  mode 
(see  Figure  7),  a  TAL  REQD  message  will 
appear  on  the  crewmember's  display.  In  order 
to  complete  transfer  alignment,  the 
instructor  must  issue  a  TAL  RECOGNIZED 
command.  This  is  done  by  entering  3  CR 
from  the  RUN  menu.  Once  entered,  TAL  is 
completed  and  the  missiles  enter  Fine 
Alignment  (FA)  mode.  If  the  TAL  RECOGNIZED 
command  is  entered  before  either  the  TAL  mode 
is  reached  or  the  TAL  REQD  message  appears, 
it  will  be  ignored  and  have  no  affect  on  the 
system. 

The  Terrain  Avoidance  (TA)  mode  affects 
the  width  of  the  sector  that  appears  in  the 
radar  video  display.  There  are  two  values 
for  the  mode:  1  (OFF)  and  2  (ON).  Upon 
entering  4  CR  ,  (see  Figure  7)  the  current 
value  of  TA  will  be  displayed  along  with  a 
prompt  for  a  new  value.  Entering  a  value 
other  than  1  or  2  will  result  in  an  error 
message  and  prompt  for  new  input. 

The  current  TA  mode  is  also  displayed  on 
the  console  below  the  Fault  Injection  line 
following  the  Pilot  Steering  Mode  display. 

Any  change  in  the  TA  mode  will  be  reflected 
on  this  display.  At  the  start  of  the 
training  session,  the  TA  mode  will  default  to 
OFF. 

The  FREEZE  command  (see  Figure  7)  allows 
the  Instructor  to  halt  the  training  session 
at  any  point  without  destroying  the  validity 
and  consistency  of  the  simulation.  The 
complete  session  can  be  frozen  at  an  Instant 


in  time,  allowing  the  instructor  to  go  to  the 
crewmembers  and  point  out  a  sequence  of 
operations  or  recent  commands  that  were  in 
error  or  inadequate. 

The  instructor  "freezes"  the  session  by 
entering  5  CR  .  At  that  Instant,  all 
training  and  simulation  procedures  for 
crewmember  activity  will  halt.  The 
instructor  may  Interact  verbally  with  the 
crewmembers  and  advise  them  of  any  problems 
they  may  be  experiencing. 

Once  the  instructor  has  finished  his 
verbal  instruction  during  the  "freeze",  he 
can  then  resume  the  simulation  at  precisely 
the  point  at  which  it  was  stopped,  and  the 
training  session  can  continue  as  planned  from 
that  point. 

To  continue  a  "frozen"  session,  the 
instructor  enters  6  CR  (see  Figure  7).  The 
training  session  resumes  from  the  exact  point 
at  which  it  was  frozen. 

POST  RUN  MODE 

The  major  purpose  of  the  Post-Run  Mode 
(see  Figure  4)  is  to  provide  a  limited  review 
of  the  currently  defined  working  scenario. 

The  instructor  may  specify  an  arbitrary  point 
in  the  scenario  as  a  start  time  for  review. 
Once  selected,  he  may  begin  a  review  of  the 
scenario  by  activating  the  RESUME  command. 

When  the  review  is  in  progress,  the 
instructor  may  stop  the  review  at  any  time. 
This  will  enable  him  to  make  any  notes  of  the 
scenario  without  missing  critical  material  or 
allow  him  to  define  a  new  starting  point. 
Thus,  when  used  in  conjunction  with  defining 
start  points  and  resuming  the  review  process, 
a  particular  sequence  can  be  replayed  many 
times. 

POST  RUN  COMMANDS 

Four  commands  are  available  during  the 
Post-Run  mode  and  are  presented  in  menu  form 
in  Figure  10.  Only  the  DEFINE  START  POINT 
comound  requests  additional  input  from  the 
instructor,  and  none  of  the  commands  invoke  a 
second  level  of  command  menu. 


POST- 

RUN  MENU 

1) 

DEFINE  START  POINT 

2) 

FREEZE 

3) 

A  \ 

RESUME 

RETURN  TO  EXECUTIVE 

Figure  10 

Post-Run  Master  Menu 
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Activated  by  a  1  CR  ,  the  system  will 
display  the  current  start  time  (which 
defaults  to  the  beginning  of  the  flight 
plan).  The  Instructor  will  then  be  prompted 
for  a  new  start  time  and  must  enter  a  value 
that  is  between  the  start  and  end  of  the 
flight  scenario.  If  he  enters  an  illegal 
value,  an  error  will  be  flagged  and  he  will 
be  requested  for  new  input.  Once  a  new  start 
time  is  successfully  entered,  it  will  become 
the  point  at  which  the  review  process  will 
start  when  the  RESUME  command  is  activated. 

The  FREEZE  command  is  only  meaningful 
during  the  review  process.  Activated  by  a 
keystroke  sequence  of  2  CR  ,  this  command 
will  stop  the  reviewing  process. 

When  RESUME  command  is  activated,  the 
reviewing  process  will  be  started  or 
restarted.  The  scenario  will  begin  at  the 
currently  defined  start  point,  or  freeze 
point  if  the  session  was  frozen,  and  will 


continue  along  as  though  the  aircraft  was 
following  the  flight  path  perfectly.  Radar 
video  will  correspond  to  the  exact  flight 
path  with  MFD  formats  available  for 
inspection.  No  on-line  Interaction  of 
crewmember  commands  will  be  available  except 
for  radar  video  presentation  format.  Once 
the  post-run  review  is  completed,  the 
Instructor  can  return  to  the  exective  menu  by 
inputtng  a  4  CR  command.  Activation  of  this 
command  will  clear  the  POST-RUN  master 
command  menu  from  the  display  and  control 
will  return  to  the  EXECUTIVE  command  menu. 

HARDWARE 

The  PIT  has  been  designed  in  modules  to 
facilitate  modifications  as  the  OAS  Itself 
changes  and  as  additional  OAS  trainer 
capabilities  are  identified.  The  five 
modules  —  independent  subsystems  —  of  the 
hardware  configuration  are  depicted  in  Figure 
4  and  described  below.  They  Illustrate  the 


MAINFRAME  PROCESSING  SUBSYSTEM  BLOCK  DIAGRAM 


f'K  I  Ml  S',0 

MifitniMi'iirtii 

'  IIM 


Figure  12 


"8 


use  of  off  the  shelf  equlpaent. 

The  Mainframe  Processing  Subsystem  (MPS) 
consists  of  a  minicomputer  with  disk  drive, 
magnetic  tape  drive >  and  system  console  (see 
Figure  11).  The  disk  Is  used  for  real-time 
storage  and  retrieval  of  radar  Imagery  data; 
the  magnetic  tape  Is  used  for  transfer  of 
software  and  updates  of  terrain  data. 

The  MPS  Is  the  central  processor  for  the 
PTT.  It  Is  responsible  for  logical  and 
numerical  processing  plus  control  of  all 
other  subsystems.  The  PTT  Software  runs  on 
the  MPS. 

The  virtual  console  Is  a  CRT  required  for 
the  PRIMOS  Operating  System.  The  disk  drive 
Is  required  for  the  operating  system,  for 
software  and  data  file  storage,  and  for 
real-time  access  to  display  flies  for 
synthetic  radar  Imagery  updates.  The 
magnetic  tape  drive  is  required  for  loading 
terrain  files  and  software  updates  on  the 
operational  systems  In  the  field.  All 
peripherals  are  standard  PRIME  equipment. 

All  peripheral  devices  on  the  PRIME  550 
CPU  (disk  drive,  magnetic  tape  drive,  and 
virtual  console)  have  standard  Interfaces 
supported  by  PRIME  hardware  and  software. 

The  mainframe  processing  subsystem 
capabilities  are  listed  below  for  the  PRIME 
550  CPU. 

PRIME  550  CPU 

32  Bit  CPU  Architecture 

128  Registers 

512  K  Byte  Error  Correcting  Code  Main 
Memory  (expandable  to  2  M  Byte) 

1  K  Word  Cache 

Single/Double  Precision  Floating  Point 
INSTRUCTOR'S  CONSOLE  SUBSYSTEM  (ICS). 

The  Instructor's  Console  Subsystem  (ICS) 
serves  as  a  system  monitor  and  control 
Interface  for  the  Instructor.  It  provides 
for  Interchange  of  textual  data  with  the  Part 
Task  Trainer  control  programs  executing  in 
the  MPS.  The  following  data  are  displayed  at 
the  Instructor's  Console  on  an  alphanumeric 
CRT  display: 

Flight  Parameters 

Mission  Data 

Status  of  OAS-Emulated  Subsystems 

Operational  Faults 

Alphanumeric  data  and  control  codes  are 
Input  through  a  keyboard. 

Functions  which  can  be  exercised  are: 


Trainer  Session  Control 

Alteration  of  Flight  Parameters 

Modification  of  OAS-Emulated  Subsystems 
Status 

Fault  Seeding 

Recovery  from  Operational  Faults 

The  ICS  Is  a  SOROC  IQ  140  CRT  terminal  (See 
Figure  12). 

GROWTH  POTENTIAL 

We  realized  that  If  we  did  not  build  a 
growth  capability  Into  the  basic  PTT  design, 
we  would  start  at  square  one  again  If  furture 
considerations  required  an  expansion  of  PTT 
capabilities.  Therefore,  growth  potential 
was  a  high  priority  consideration  throughout 
the  entire  program.  This  requirement  for 
growth  potential  spilled  over  Into  the  design 
of  the  Instructor  station  as  well. 

First,  the  Instructor  station  EXECUTIVE 
software  was  designed  with  this  growth 
capability  In  mind.  For  example,  we  realized 
that  we  would  eventually  have  to  Include  more 
malfunctions  In  the  PTT.  Therefore,  the 
Instructor  software  had  to  be  able  to  handle 
these  additional  malfunction  requirements  as 
well.  Therefore,  the  EXECUTIVE  software 
design  Included  a  group  of  unused 
Input/output  flags.  Thus,  when  an  addition 
Is  made  to  the  PTT  software,  the  EXECUTIVE 
routines  will  not  have  to  change.  This  will 
greatly  reduce  the  time  and  cost  of  software 
development . 

The  system  hardware  also  lends  Itself  to 
expansion.  A  30%  growth  capability  was 
designed  Into  the  PTT  mainframe  computer.  If 
expansion  requires  more  than  the  planned  pad, 
we  can  add  a  50%  memory  capability  by 
purchasing  a  $12,000  memory  board. 

The  SOROC  IQ  140  terminal  (see  Figure  12) 
also  provides  us  with  tremendous  growth 
potential.  We  have  not  even  begun  to  take 
advantage  of  the  special  functions  available 
on  the  SOROC. 

Finally,  the  Instructor's  command  formats 
and  displays  are  Ideally  suited  to  expansion. 
The  single  keystroke  command  format  can 
easily  handle  doubledigit  or  even  triple¬ 
digit  commands.  In  addition,  the 
instructor's  displays  can  be  easily  changed 
to  meet  growth  requirements.  For  example.  If 
we  were  to  add  more  faults  than  could  fit  on 
the  Fault  Injection  Menu  (Figure  8),  we  could 
simply  scroll  this  menu  just  as  the 
destination  tables  are  already  scrolled  in 
the  present  configuration. 
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SUMMARY 


The  B-52  Offensive  Avionics  System  (OAS) 
Part  Task  Trainer  (PTT)  is  a  trainer  that  is 
designed  to  focus  on  procedures  training; 
specifically  those  procedures  unique  in 
operation  of  the  new  B-52  Offensive  Avionics 
System.  With  emphasis  on  procedures 
training,  the  instructor  plays  a  major  role 
in  the  training  of  the  students.  The  OAS  PTT 
was  a  two-user  interface,  one  for  the 
crewmembers  and  one  for  Che  instructor.  The 
instructor's  interface  is  designed  to  free 
the  instructor  to  Interact  with  the  students 
at  all  times.  Specification  requirements  of 
the  instructor  station  included;  ease  of 
operation,  rapid  scenario  setup,  access  to 
students,  easy  student  and  system  monitoring, 
the  growth  capability.  With  these 
requirements  in  mind,  an  instructor  station 
and  interface  was  designed  to  make  the 
instructor's  job  simple  and  effective. 
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TRAINING  EFFECTIVENESS  EVALUATIONS  OF 
SIMULATOR  INSTRUCTIONAL  SUPPORT  SYSTEMS 

DR.  STEVE  R.  OSBORNE  AND  DR.  GEORGE  W.  MENZER 
Allen  Corporation  of  America 


Substantial  increases  in  simulator 
technology  have  dramatically  increased  the 
scope  and  potential  of  simulator  training. 
Ultimate  simulator  training  effectiveness, 
however,  is  not  only  a  function  of  a 
device's  capability  to  simulate  training 
tasks  accurately,  but  also  of  its  ability  to 
operate  as  an  effective  instructional  tool. 
Its  effectiveness  as  an  instructional  tool 
depends  upon  Instructor/Cperator  Station 
(IOS)  factors  such  as  instructor /operator 
workload,  performance  monitoring  and  evalua¬ 
tion  capabilities,  and  training  task/mission 
set-up.  Effective  design  and  proper  utili¬ 
zation  of  the  IOS  can  affect  the  potential 
and  achieved  effectiveness  of  simulator 
training. 

Advances  in  simulator  technology,  and 
atterpts  to  incorporate  state-of-the-art 
instructional  technology,  have  resulted  in 
increasingly  ccnplex  and  sophisticated 
instructor /operator  stations.  Unfortu¬ 
nately,  this  application  of  new  technology 
has  not  always  led  to  advances  in  instruc¬ 
tional  effectiveness  of  efficiency.  This 
shortcoming,  and  the  potential  importance  of 
effective  IOS  functioning,  highlight  the 
importance  of  evaluating  IOS  design  and 
innovations  in  simulator  instructional 
support  systems.  However,  accurately 
assessing  the  effectiveness  of  simulator 
instructional  support  systems  poses  a  set  of 
problems  that  often  are  not  solved  by 
traditional  approaches  to  evaluating  simu¬ 
lator  training  effectiveness. 

This  paper  discusses  sane  of  the 
problems  encountered  in  attempting  to 
evaluate  the  effectiveness  of  simulator 
instructional  support  systems  and  provides 
seme  potential  solutions  to  these  problems. 
Three  areas  are  addressed;  selection  of  a 
suitable  evaluation  model  and  sensitivity  of 
performance  measures;  evaluation  objectives; 
and  instructor  training. 

Evaluation  Models 

Additions  or  modifications  to  training 
systems  usually  most  be  justified  by  showing 
that  they  either  increase  training  effective¬ 
ness  and/or  that  they  somehow  reduce 
training  costs  while  maintaining  some  given 
level  of  ef fectiveness .  This  is  especially 
true  if  the  addition  or  modification  repre¬ 
sents  a  prototype  or  proof-of-concept 
training  device  or  instructional  aid. 

In  general,  evaluation  models  used  to 
assess  training  effectiveness  can  be  grouped 


into  one  of  two  categories:  analytical 
models  and  experimental  models.  Analytical 
models  span  a  broad  variety  of  procedures 
ranging  from  simple  questionnaires  and 
opinion  surveys  to  well-developed,  highly 
structured  rating  scale  evaluations. 

Analytical  models  usually  do  not  involve 
direct  measures  of  student  or  instructor 
performance.  Instead,  they  rely  upon  esti¬ 
mates,  judgements,  or  opinions  that  are 
based  on  direct  observation  or  experience. 
Experimental  models  rely  on  direct  measure¬ 
ment  of  performance.  They  normally  involve 
carparing  the  performance  of  one  group  of 
students  trained  with  the  addition/modifi¬ 
cation  to  the  performance  of  another  group 
of  students  who  are  trained  without  the 
addition  or  modification.  Under  this  model, 
differences  in  performance  can  be  attributed 
to  the  influence  of  the  instructional  addi¬ 
tion  or  modification  provided  that  other 
factors  that  influence  performance  are  held 
constant  for  both  groups  of  students. 

A  transfer  of  training  evaluation  is 
representative  of  the  experimental  model  in 
which  the  amount  of  training  required  to 
attain  proficiency  with  the  actual  equipment 
is  related  to  the  amount  of  previous 
(simulator)  training.  Transfer  of  training 
studies  allow  simulator  training  effective¬ 
ness  to  be  expressed  as  the  amount  of  actual 
equipment  training  that  can  be  saved  by 
simulator  training.  Such  studies  also  allow 
calculation  of  potential  cost  savings  that 
can  be  achieved  through  simulator  training. 

Experimental  comparisons,  such  as 
transfer  of  training  studies,  have  become 
the  standard  for  assessing  simulator 
training  effectiveness.  There  has  been  a 
natural  tendency,  therefore,  to  apply  these 
same  experimental  models  to  assess  the 
effectiveness  of  simulator  instructional 
support  systems  such  as  instructor /operator 
stations,  special  instructional  features, 
and  simulator  instructional  capabilities. 
Fran  a  methodological  standpoint,  experi¬ 
mental  models  represent  the  preferred 
approach  to  assessing  training  effective¬ 
ness.  However,  there  are  at  least  two 
factors  uhat  limit  the  adequacy  of  this 
approach  for  evaluating  simulator  instruc¬ 
tional  support  systems. 

The  first  factor  concerns  the  expected 
contribution  of  the  instructional  system 
additicn/modification  to  training  ccrrpared 
to  the  contribution  made  by  the  overall 
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training  system.  If  the  relative  increment 
in  training  effectiveness  is  stall,  then  it 
may  not  be  detected  on  the  overall  system 
level,  i.e. ,  in  terms  of  student  perform¬ 
ance.  For  exanple,  student  performance 
typically  is  affected  by  multiple  ccnponents 
of  the  training  system  such  as  academic, 
classroom,  simulator,  and  actual  equipment 
training.  In  most  cases,  the  instructional 
support  system  of  a  simulator,  compared  to 
the  overall  training  system,  represents  a 
modest  influence  on  student  performance. 
Such  small  effects  require  very  sensitive 
performance  measures  and  a  level  of  experi¬ 
mental  control  and  rigor  that  usually  is  not 
available  for  evaluations  conducted  in 
operational  settings. 

The  second  factor  concerns  using 
instructor-assigned  grades  as  a  measure  of 
student  performance.  Although  instructor 
grades  frequently  are  used  in  training 
effectiveness  evaluations,  they  nornally  are 
not  very  sensitive  to  actual  differences  in 
student  performance.  Instructor  grades  are 
used  more  to  manage  student  progress  through 
the  training  program  than  they  are  to 
measure  performance.  Instructor  grades  also 
tend  to  be  norm-referenced,  i.e. ,  they  are 
assigned  relative  to  the  average  performance 
of  other  students  of  oonparable  training  and 
experience.  Although  grades  serve  their 
intended  purpose,  they  alone  usually  are  not 
sufficient  to  detect  differences  in  perform¬ 
ance,  especially  if  those  differences  are 
small. 

The  solution  of  the  first  problem 
requires  that  the  evaluation  be  conducted  at 
a  level  caimensurate  with  the  expected 
increment  in  training  effectiveness.  If  the 
change  or  addition  to  training  represents  a 
major  impact  on  training,  then  it  may  be 
sufficient  to  assess  that  change  at  a  molar 
level,  such  as  by  measuring  differences  in 
student  performance.  On  the  other  hand,  if 
the  change  or  modification  represents  a 
small  inpact  then  it  will  be  necessary  to 
assess  that  change  at  a  more  molecular 
level.  This  may  require  a  change  in  the  way 
training  effectiveness  evaluations  are 
conducted.  Experimental  comparisons  of 
student  performance  nay  need  to  be  replaced 
or  supplemented  with  more  analytical  compar¬ 
isons.  For  exanple,  changes  in  the  instruc¬ 
tional  features  of  a  simulator,  which  repre¬ 
sent  a  snail  change  in  the  overall  training 
system,  could  be  evaluated  by  establishing  a 
list  of  behavioral  objectives  that  the  new 
instructional  capability  should  achieve. 
The  evaluation  would  then  assess  the  extent 
to  which  those  objectives  were  met.  Such 
objectives  should  be  a  natural  part  of  the 
design  process  itself  and  they  should 
reflect  the  original  purpose  for  developing 
or  modifying  a  particular  instructional 
feature  or  capability.  Establishing  a  list 


of  behavioral  objectives  would  allow  a  clear 
and  specific  statement  to  be  made  about  the 
expected  results  of  a  particular  simulator 
feature  and  also  would  provide  a  clear  basis 
for  assessing  the  extent  to  which  those 
results  were  achieved.  Ibis  information 
could  be  used  to  evaluate  the  merits  of  a 
given  simulator  feature,  regardless  of  how 
snail  a  corrponent  that  feature  was,  relative 
to  the  overall  training  system.  An  assess¬ 
ment  of  behavioral  objectives  also  could  be 
viewed  within  the  context  of  a  broader  scale 
evaluation  of  student  performance. 

The  second  problem,  i.e.,  sensitivity 
of  instructor  grades  can  be  addressed  in  a 
number  of  ways.  First,  a  set  of  objective 
measures  of  student  performance  could  be 
developed .  However,  the  time  and  cost 
requirements  to  develop  good  performance 
measures  often  exceeds  the  resources  avail¬ 
able  for  training  effectiveness  evaluations. 
Second,  instructors  could  be  asked  to  record 
actual  deviations  in  performance  for  a 
selected  set  of  parameters  judged  to  be 
especially  critical  to  successful  perform¬ 
ance  of  the  task  being  graded.  This  would 
provide  sane  additional  performance  infor¬ 
mation  for  the  purpose  of  the  evaluation. 
It  also  would  allcw  subsequent  instructor 
ratings  to  be  based,  in  part,  on  selected 
aspects  of  student  performance.  Third,  the 
standard  four-point  grading  scale  used  by 
instructors  could  be  expanded  to  a  seven- 
point  scale  by  inserting  intermediate  points 
between  each  original  scale  point.  This 
would  increase  the  potential  sensitivity  of 
the  middle  part  of  the  scale  which  is  most 
frequently  used,  but  it  would  not  signifi¬ 
cantly  alter  the  basic  format  of  the  scale. 
Therefore,  instructors  still  would  have  a 
scale  they  are  familiar  with,  but  one  that 
has  greater  sensitivity.  Finally,  automated 
performance  capabilities  of  simulators 
should  be  utilized  if  they  are  available. 

Evaluation  Objectives 

Related  to  the  topic  of  evaluation 
models  is  the  issue  of  evaluation  objec¬ 
tives.  Although  the  overall  goal  of  an 
evaluation  may  be  to  assess  the  training 
effectiveness  or  efficiency  of  a  simulator 
instructional  support  system,  more  specific 
evaluation  objectives  should  be  formulated. 
Specific  evaluation  objectives  are  especi¬ 
ally  important  for  evaluating  instructional 
support  systems  because  the  specific  proce¬ 
dures  for  evaluating  such  systems,  including 
dependent  variables,  are  not  as  veil  defined 
as  they  are  for  evaluations  of  total 
simulator  systems.  Specific  evaluation 
objectives  '  can  provide  a  conceptual 
framework  for  the  development  of  specific 
evaluation  procedures.  Clearly  stated, 
specific  evaluation  objectives  also  can  be 
helpful  in  selecting  an  evaluation  model  and 
for  guiding  the  development  of  measures  of 
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instructional  effectiveness. 

The  development  of  evaluation  objec¬ 
tives  can  take  many  forms,  however,  sane 
criteria  that  objectives  should  satisfy  are 
as  follows.  First,  objectives  should  be 
based  on  the  overall  purpose  of  the 
evaluation.  Objectives,  therefore,  should 
be  stated  so  that  their  fulfillment  will 
provide  answers  to  the  questions  that 
originally  generated  the  evaluation. 
Second,  objectives  should  be  operationally 
meaningful  in  the  sense  that  their  fulfill¬ 
ment  will  provide  useful  information. 
Third,  objectives  should  be  stated  clearly. 
Finally,  objectives  should  be  testable 
within  the  operational  context  in  which  the 
evaluation  will  be  conducted. 

Instructor  Training 

Another  important  factor  in  conducting 
an  accurate  evaluation  of  any  instructional 
support  system  involves  instructor/operator 
training.  A  device  must  be  used  correctly, 
and  with  sufficient  knowledge  of  its  limita¬ 
tions  and  capabilities,  to  achieve  an 
accurate  assessment  of  its  true  effective¬ 
ness.  Instructor  training  also  plays  a  key 
role  in  user  acceptance  of  new  training 
systems.  Unfortunately,  many  evaluations 
begin  without  adequate  instructor/operator 
training. 

The  solution  to  the  problem  of 
instructor  training  is  straightforward. 
Training  effectiveness  evaluations  of  simu¬ 
lators  and  their  instructional  support 
systems  should  not  be  started  until  the 
instructors/operators  who  are  going  to  use 
them  are  adequately  trained.  The  nature  of 
such  training  should  be  threefold.  First, 
instructor /operators  need  to  know  hew  to 
operate  the  simulator.  Second,  they  need 
to  be  trained  in  hew  to  effectively  utilize 
the  instructional  capabilities  of  the  simu¬ 
lator.  Third,  they  need  to  be  trained  with 
the  procedures  that  are  going  to  be  used 
during  the  evaluation. 

Theoretically,  instructor  training 
should  be  a  siitple  task  to  ccrplete. 
Hcwever,  it  often  poses  a  nunber  of 
problems,  including  the  time  required  to 
train  all  of  the  instructors,  scheduling 
instructor  training  around  the  existing 
training  schedule,  and  rotation  of  instruc¬ 
tors  in  and  out  of  the  training  command. 
Many  of  these  problems  could  be  solved  by 
selecting  a  set  of  instructors  who  are  not 
expected  to  transfer  away  from  the  training 
squadron  until  after  the  evaluation  is 
completed.  Having  a  smaller  group  of 
instructors  participate  in  the  evaluation 
would  simplify  the  instructor  training  task 
and  would  perhaps  allow  enough  time  to 
provide  them  with  some  individualized 
on-the-job  tutorials.  Alternatively,  using 
a  subset  of  available  instructors  will 


reduce  the  nunber  of  students  that  can  be 
followed  at  any  given  time  and,  hence,  will 
increase  the  duration  of  the  evaluation.  It 
also  may  cause  sane  problems  for  scheduling 
simala tor  training  events. 

Svrnrary 

Conducting  training  effectiveness  eval¬ 
uations  of  simulator  instructional  support 
systems  involves  all  of  the  problems  gener¬ 
ally  associated  with  assessing  simulator 
effectiveness  and  poses  sane  special 
problems  as  well.  This  paper  addressed  sane 
of  these  problems  and  discussed  some  poten¬ 
tial  solutions. 

lusted  below  are  sane  guidelines  that 
nay  help  in  designing  and  conducting 
training  effectiveness  evaluations  of 
instructional  support  systems.  These  guide¬ 
lines  incorporate  same  of  the  problem 
solutions  discussed  above  as  well  as  sane 
reoarmendations  not  discussed. 

-  Ensure  that  the  system  to  be  evaluated  is 
sufficiently  developed  and  debugged  before 
the  evaluation  begins. 

-  Develop  specific  evaluation  objectives 
that  can  be  addressed  with  the  resources 
available  to  conduct  the  evaluation. 

-  Select  or  develop  an  evaluation  model  that 
will  allcw  evaluation  objectives  to  be 
satisfied. 

-  Ensure  that  the  evaluation  model  is 
appropriate  for  the  size  of  the  training 
effect  expected  to  be  detected  by  the 
evaluation. 

-  Select  or  develop  a  set  of  measures  that 
together  will  allow  all  evaluation  objec¬ 
tives  to  be  addressed. 

-  Ensure  that  measures  of  performance  are 
sensitive  enough  to  detect  differences 
attributable  to  using  the  instructional 
system  being  evaluated. 

-  Develop  a  conceptual  plan  for  relating 
measures  of  performance  to  evaluation 
objectives  (this  is  especially  important 
for  evaluations  that  use  analytical  models) . 

-  Ensure  that  instructors/operators  tire 
adequately  trained  to  operate  the  simulator, 
and  to  utilize  the  capability  of  the  in¬ 
structional  system  being  evaluated. 
Instructors/operators  also  should  be  trained 
with  the  evaluation  procedures  to  be 
followed. 

-  Monitor  daily  the  conduct  of  the  evalua¬ 
tion  and  data  collection  procedures. 
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Following  these  guidelines  will  not 
guarantee  a  successful  evaluation  and  they 
do  not  address  all  of  the  issues  associated 
with  conducting  a  training  effectiveness 
evaluation.  However,  they  may  provide  sane 
assistance  in  designing  and  planning  an 
evaluation  and  they  may  help  minimize  same 
of  the  problems  encountered  by  previous 
evaluations  of  instructional  support  systems. 
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ABSTRACT 


The  increasing  technical  complexity  of  modern 
systems  require  the  application  of  new  techniques  in 
the  design  of  the  man-machine  interface.  Stress 
caused  by  information  overload  and  "underload"  will 
not  be  relieved  by  increased  training  or  better  per¬ 
sonnel  selection.  Rethinking  the  man-machine  dialogue 
is  required.  Advances  in  modeling  methodologies  offer 
promising  possibilities  for  the  future;  3-D  Computer- 
Aided  Design  (CAD)  systems  offer  solutions  that  are 
available  today.  Application  of  one  such  system,  the 
Enhanced  Interactive  Design  System  (EIDS) ,  is  dis¬ 
cussed  with  reference  to  its  use  in  actual  weapons 
system  design. 


INTRODUCTION 

Ever  more  complex  technological  demands 
for  ever  more  accurate  weapons  systems  have 
made  these  systems  so  complex  that  the  oper¬ 
ating  personnel  are  being  pushed  toward  their 
operational  performance  limits.  Neither  pro¬ 
longed  training  nor  increased  personnel  selec¬ 
tivity  can  ensure  the  right  solutions  to  these 
pressing  problems.  The  use  of  human  engi¬ 
neering  principles  to  define  the  man-machine 
dialogue  and  the  man-machine  Interface  has  long 
been  stressed  as  the  solution  to  the  need  for 
shorter  training  times  and  lower  qualification 
profiles,  with  better  operating  results.  How¬ 
ever,  appropriate  human  engineering  principles 
have  yet  to  be  adequately  applied. 

Human  factors  engineering  considers  a 
large  area  that  Includes  not  only  anthropomet¬ 
ries,  mechanical,  and  chemical  (chemo-physi- 
cal)  stress  factors  but  also  the  whole  range  of 
man-machine  interactions.  These  factors  im¬ 
pact  the  man-machine  dialogue  through  indica¬ 
tors,  controls,  and  visual  and  auditory  infor¬ 
mation  from  the  environment  and  also  Include 
other  sensory  data  and  Information  that  are  not 
directly  related  to  the  operator's  task,  such 
as  movement,  especially  accelerations  and  vi¬ 
brations,  sound,  noise,  temperature,  and  the 
whole  microclimate.  Another  factor  in  the  man- 
machine  dialogue  Is  the  proprioceptive  Infor¬ 
mation,  l.e.,  the  Information  regarding  the  lo¬ 
calization  of  the  body  and  its  extremities  In 
space.  Its  movement,  etc. 

In  today's  human  engineering  community, 
our  efforts  must  no  longer  be  primarily  ori¬ 


ented  toward  further  increases  in  the  rate  of 
data  output.  The  way  toward  higher  efficiency 
is  increasingly  dependent  on  how  well  we  suc¬ 
ceed  in  reducing  stress.  Man  has  become  the 
weakest  link  in  the  chain.  This  is  mostly  be¬ 
cause  engineers  and  system  designers  have  al¬ 
ways  tried  to  solve  the  technical  problems 
first.  Because  of  their  mechanistic  blinders, 
they  have  hardly  ever  confronted  man-related 
problems  unless  forced  to.  Those  human  factors 
that  were  recognized  and  respected  were  the 
most  primitive  and  easiest  to  capture,  e.g., 
anthropometries  (at  least  I  have  no  other  ex¬ 
planation  why,  for  a  lot  of  people,  ergonomics 
is  still  understood  as  a  synonym  for  anthropo¬ 
metries).  Man  is  much  more  complex  than  simply 
a  set  of  measurements.  A  lot  of  physiological 
and  psychological  knowledge  has  been  collected 
during  the  past  50  years,  but  little  of  this 
has  been  directly  applied  to  workstation  de¬ 
sign.  Actually,  having  approached  the  limits 
of  man's  capabilities,  we  largely  lack  the 
methods  for  applying  this  knowledge  to  the  en¬ 
gineering  of  a  modern  workstation. 


STRESS  AND  INFORMATION  LOAD 


One  of  the  major  tasks  for  human  factors 
specialists  is  stress  reduction.  In  order  to 
achieve  optimum  results  in  stress  reduction,  we 
have  to  work  with  both  the  chemo-physical 
stress  factors  and  the  proprioceptive  informa¬ 
tion.  Only  a  coordinated  effort  on  all  areas 
can  assure  good  results,  since  otherwise  Iso¬ 
lated  "peaks"  of  stress  can  affect  the  results 
of  all  the  efforts.  (It  Is  often  noted  that  an 
Isolated  disturbance  in  an  otherwise  undis- 


turbed  environment  will  generally  be  much  more 
noted  and  have  more  effect  than  if  surrounded 
by  a  higher  level  of  other  disturbances.) 

In  order  to  achieve  a  better,  more  ef¬ 
fective,  and  more  operational  interface  by 
structuring  the  man-machine  dialogue,  we  must 
have  as  a  goal  the  reduction  of  stress.  Stress 
itself  is  in  some  respects  linked  to  informa¬ 
tion.  There  can  be  all  kinds  of  information, 
not  only  those  regarding  the  task  itself. 
Stress,  from  this  point  of  view,  can  be  caused 
by  both  under-information  and  over-informa¬ 
tion.  If  one  gets  too  little  information,  one 
instinctively  starts  to  worry  and  tries  to  fill 
the  gap,  searching  for  more  information.  This 
applies  not  only  to  information  needed  con¬ 
sciously  for  the  task,  but  as  well  to  subcon¬ 
sciously  needed  information  regarding  the  en¬ 
vironment  and  oneself,  e.g.,  proprioceptive 
information.  Over-information  as  well  can  pro¬ 
voke  stress  and  thus  hinder  the  proper  accom¬ 
plishment  of  a  task.  We  must  therefore  strive 
to  suppress  all  unnecessary  and  superfluous  in¬ 
formation.  Stress  reduction  by  structuring  the 
man-machine  dialogue  goes  beyond  the  anthropo- 
metrical  positioning  of  controls  based  on  the 
criteria  of  reach  and  operability  and  beyond 
positioning  of  indicators,  in  order  to  read 
them  better  and  more  quickly.  Structuring  the 
man-machine  interface  is  a  very  complex  proc¬ 
ess,  which,  under  given  time-cost  constraints, 
and  manpower  constraints,  requires  new,  more 
effective  tools  and  methods. 


THE  ROLE  OF  MODELS  IN  SYSTEM  DESIGN 

Many  different  tools  have  been  developed 
that  attempt  to  handle  the  complexity  of  param¬ 
eter  interdependence  in  the  anthropometrical 
field.  Man-models  have  been  developed  and  suc¬ 
cessfully  appied  in  many  different  areas. 
Techniques  include  bidimensional  approaches, 
the  simplest  being  the  Dreyfuss  scale,  the 
Bosch  drawing  aids  developed  by  Dr.  Jenik  of 
Darmstadt  University,  and  other  very  special 
application-oriented  approaches,  like  the  com¬ 
puter-based  design  for  car  driver's  worksta¬ 
tions  by  Volkswagen,  the  Reach  model  for  astro¬ 
nauts  used  by  NASA,  and  more  complex  man-models 
such  as  SAMMIE,  BOEMAN,  CAR  (Edwards,  et  al., 
1976),  CAPE  (Bittner,  1975),  COMBIMAN  (Evans, 
1972;  Kroemer,  1973;  Dillhoff,  Evans,  and 
Krause,  1974),  and  HOS  (Strieb,  1974;  Lane, 
Wherry,  Strieb,  1977).  It  is  tempting  to  con¬ 
sider  developing  even  more  complex  computer- 
based  man  models,  where  many  more  parameters, 
like  comfort  and  maximum  limits  of  movement  and 
joints,  forces,  reaction  time,  manipulative 
precision,  resonance  frequencies  of  organs, 
etc.,  would  be  considered.  The  data  from  these 
models  could  be  stored  and  automatically 
studied  in  relationship  to  postures  within  the 
working  environment. 


Such  models  are  feasible  and  given  the 
increasing  power  of  modern  computer  systems 
will  be  developed  as  man  reaches  toward  more 
perfectionistic  levels  of  elaboration  of  such 
man  models.  However,  because  of  economic  con¬ 
siderations  and  pending  the  development  of  such 
elaborate  models,  we  must  be  satisfied  with 
less  perfectionistic  approaches  and  make  opti¬ 
mum  use  of  available  low-cost  computer-aided 
design  systems,  available  specialized  man¬ 
power,  and  last  but  not  least  common  sense. 


COMPUTER-AIDED  DESIGN  SYSTEMS 

Today  three-dimensional  computer-aided 
design  systems  (3-D  CAD  systems)  are  available 
that,  for  a  reasonable  price,  can  run  on  the 
medium-sized  computers  already  available  in 
many  institutions  and  companies.  The  Enhanced 
Interactive  Design  Systems  (EIDS),  a  CAD  soft¬ 
ware  package  that  we  use  for  engineering, 
architectural  design,  and  human  engineering  of 
man-machine  interfaces,  can  run  even  on  small 
16  bit  computers  such  as  an  HP1000.  With  such 
a  CAD  system,  we  have  a  tool  at  hand  which  can 
be  used  in  developing  interesting  alternative 
approaches  to  workstation  layout. 

A  tool  is,  of  course,  only  worthwhile  if 
it  is  both  helpful  and  cost  effective.  Al¬ 
though  initially  expensive,  the  cost  effec¬ 
tiveness,  i.e.,  the  return  on  investment,  of  a 
3-D  CAD  system  is  favorable  if  you  take  into 
account  that  through  its  use  we  are  really  mul¬ 
tiplying  the  output  achievable  from  the  rather 
limited  human  factors  manpower  available  in  the 
system  design  field.  The  rate  of  return  on 
investment  is  very  much  dependent  on  the  use 
one  makes  of  the  data  base  once  it  is  devel¬ 
oped.  In  our  engineering  designs,  we  make  re¬ 
peated  use  of  the  data  in  designing  (and  rede¬ 
signing)  the  man-machine  interface.  Once  we 
have  established  the  linkages  between  the  data, 
the  engineers  can  continually  redesign  the 
"black  boxes"  and  their  contents  to  optimize 
the  system's  functionality. 

State-of-the-art  CAD  systems  allow  a 
total  three-dimensional  description  of  objects 
and  structures.  For  most  crewstation  design 
situations,  only  a  true  3-D  solid  modeling  CAD 
system  will  su^ice.  So-called  2^-D  systems 
are  not  of  much  help  nor  are  wire-frame  3-D 
systems  that  lack  the  hidden  line  and  hidden 
surface  capabilities  of  solid  modeling  sys¬ 
tems.  The  data  structures  of  a  3-D  solid 
modeling  system  are  such  that  the  topological 
description  of  the  real  object  is  stored  in  the 
computer.  All  imaginable  views,  planes,  iso- 
mp trie,  or  perspective,  as  well  as  cutaways  are 
just  "aspects"  (in  the  l  at  in  sense  of  the 
word).  The  computer  does  not  "think"  of  the 
objects  as  planimetric  projections.  But  any 
desired  view  can  be  obtained  by  choosing  a 
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viewpoint,  a  direction  of  view  and  the  type  of 
view,  and  striking  a  few  keys  on  the  computer 
terminal's  keyboard. 

A  further  important  feature  is  the  abil¬ 
ity  to  remove  or  suppress  hidden  lines  and  hid¬ 
den  surfaces.  Only  with  such  features  can  we 
evaluate  visibility  and  parallax  problems  and 
obtain  views  of  the  objects  that  adequately  re¬ 
flect  the  real  product  or  structure  being  de¬ 
signed.  With  such  a  system,  we  can  design 
every  item  of  our  workstation  and  assemble 
these  items  and  structures  in  many  different 
ways,  trying  out  alternatives  and  testing  for 
reach  and  legibility  for  representative  popu¬ 
lations.  Since  the  system  is  a  3-0  system  and 
since  the  computer's  internal  description  is  an 
entirely  three-dimensional  topological  one,  we 
are  able  to  try  different  inclinations  of 
panels,  examine  parallax  problems  from  dif¬ 
ferent  viewpoints,  etc.  (Eichweber,  1981) 

But  besides  enabling  us  to  control  every 
detail  of  the  layout  and  enabling  us  to  examine 
different  anthropometric  conditions  and  pos¬ 
tures  for  representative  populations,  EIDS 
aids  us  in  the  process  of  organizing  the  man- 
machine  dialogue  by  enabling  us  to  try  out  dif¬ 
ferent  arrangements  of  indicators  and  con¬ 
trols.  We  can  design  and  test  symbols  and 
scales  and  develop  new  integrated  solutions  for 
the  man-machine  dialogue.  We  can  determine  the 
best  organization  for  readability.  If  the  in¬ 
dicators  move,  we  can  test  different  positions 
and  combinations  of  the  respective  symbols, 
even  in  their  extremes.  We  can  "build"  our 
workstations  and  examine  them  from  any  view¬ 
point,  test  and  discuss  them  with  others.  The 
tedious  time-consuming  and  costly  process  of 
building  mock-ups  can  in  many  cases  be  almost 
totally  omitted.  We  can  ensure  that  in  most  of 
our  man-machine  interfaces,  little  or  nothing 
will  have  to  be  changed  as  the  result  of  trials 
even  if  we  go  directly  to  prototype  construc¬ 
tion. 

For  different  workstations  involving 
similar  tasks,  we  can  design  functional 
modules,  which,  once  they  are  stored  in  the 
data  base,  can  be  reused  for  different  applica¬ 
tions.  For  standard  engineering  analyses,  we 
can  use  the  geometric  datu  in  finite  element 
stress  analysis  programs  and  for  the  numerical 
control  of  parts  machining  equipment. 

Three-dimensional  data  bases  can  also  be 
used  for  simulation  processes.  Indicator  sys¬ 
tems  can,  for  example,  be  represented  on  dis¬ 
play  screens,  linked  to  simulators  of  the  re¬ 
spective  workstation  or  weapons  system,  and 
then  be  used  together  with  appropriate  controls 
to  simulate  the  response  of  the  system  to  the 
actions  of  the  operators.  This  can  be  helpful 
for  instance  in  the  layout  and  immediate 
testing  of  crew  stations  for  airplanes,  fire 
control  systems,  traffic  controllers,  etc. 


Finally,  we  can  use  the  data  base  to  ob¬ 
tain  the  documentation  drawings  needed  for 
training  handbooks,  service  manuals,  and  il¬ 
lustrated  parts  lists.  Perspective  drawings 
and  exploded  views  are  readily-available  by¬ 
products  of  the  same  data  base.  If  one  con¬ 
siders  the  extensive  amount  of  manpower  and 
time  it  costs  to  obtain  these  with  traditional 
drafting  techniques,  it  is  obvious  that  the  re¬ 
peated  use  of  the  same  3-D  data  base  multiplies 
its  cost-effectiveness. 

At  Eichweber,  for  instance,  we  have  used 
EIDS  for  the  engineering  design  of  the  newest 
generation  of  our  Tactical  Laser  Illuminating 
Shot  Simulators  (TALISSI)  systems  for  directly 
aimed  weapons.  All  mechanical  parts  in  the 
system  were  entered  into  the  data  base.  Moving 
parts  are  simulated  in  their  various  opera¬ 
tional  positions.  Perspective  and  exploded 
view  drawings  for  documentation  are  also  pre¬ 
pared  with  EIDS. 


SLHAARY 

In  oi/r  work,  we  have  found  3-D  CAD  sys¬ 
tems  to  be  a  vital  tool  in  the  proper  human 
engineering  of  a  modern  weapons  system.  Never¬ 
theless,  a  tool  is  only  worthwhile  if  it  is 
applied  well.  Human  engineering  can  only  be 
done  well  if  all  available  knowledge  and  cri¬ 
teria  are  applied  adequately. 

To  achieve  this  goal,  we  use  interdisci¬ 
plinary  teams  and  what  we  call  the  Value  Design 
method  (Eichweber,  1981).  This  method  is  based 
on  Value  Engineering  and  helps  to  get  close  to 
the  optimum  in  both  function  and  cost.  (By  the 
word  "design"  we  mean  tc  indicate  that  man  and 
man-related  functional  considerations  are  in¬ 
herent  in  the  conceptual  design.)  This  approach 
helps  us  to  easily  integrate  specialists  into 
an  interdiscipl inary,  goal -or iented  working 
team,  with  the  rules  of  Value  Engineering  and 
Value  Analysis  serving  as  the  "rules  of  the 
game. " 
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INTRODUCTION 

The  ever-increasing  sophistication  and 
complexity  of  weapon  systems  and  the  atten¬ 
dant  rise  in  costs  are  parallel  trends  in  the 
devices  being  procured  to  support  weapon  sys¬ 
tems  training.  Compounding  the  situation  is 
the  explosion  in  simulation  technology  which 
has  led  to  development  of  sophisticated  train¬ 
ing  devices.  Often  the  devices  have  resuited 
from  the  technology  because  it  is  available 
and  not  as  a  result  of  an  analysis  defining 
the  specific  training  requirement. 

In  the  development  of  a  training  device, 
the  first  step  is  an  analysis  of  the  train¬ 
ing  requirements  which  the  device  will  satis¬ 
fy.  Over  the  years  these  analyses  have  been 
non-existent  or  when  conducted  have  taken 
various  forms  with  differing  levels  of  accur¬ 
acy  and  thoroughness.  All  too  often  the 
analyses  have  been  a  cursory  assessment  of 
the  existing  or  projected  training  require¬ 
ments.  As  a  result  the  design  and  develop¬ 
ment  of  the  devices  have  been  left  in  the 
hands  of  engineering,  computer,  and  software 
specialists.  Their  primary  orientation  and 
interest  is  the  development  of  training  hard¬ 
ware  which  operates  in  accordance  with  the 
approved  specification.  This  does  not,  how¬ 
ever,  necessarily  mean  that  the  device  satis¬ 
fies  the  training  requirements. 

The  emphasis  on  the  engineering  develop¬ 
ment  of  a  training  device  has  developed  by 
default  (i.e.,  the  lack  of  an  analysis  of  the 
training  requirements).  The  lack  of  partici¬ 
pation  by  human  factors  and  training  analysis 
personnel  in  providing  the  necessary  input 
related  to  the  training  requirements,  the 
instructor  role,  ease  of  operation,  and  in 
particular  the  design  of  the  Instructor/Oper¬ 
ator  Station  (IOS),  has  been  a  major  contri¬ 
butor  to  the  situation. 

As  a  result  training  devices  have  been 
delivered  with  lOSs  which  are  complex  and 
difficult  to  operate.  For  example,  many  IOSs 
have  multiple  data  entry  methods  such  as 
switches,  light  pens,  numeric  keyboards. 


alpha-numeric  keyboards,  multi-function 
switches,  pages  and  pages  of  CRT  information 
which  must  be  called  up  before  the  specific 
item  can  be  located,  etc.  Extensive  formal 
training  is  required  for  the  instructor/oper¬ 
ator  to  become  proficient.  Daily  use  is 
required  to  remain  familiar  with  the  opera¬ 
tion  of  the  IOS.  In  the  "real  world"  situa¬ 
tion  the  instructor/operator  may  or  may  not 
have  had  formal  training.  More  likely  he 
has  received  an  abbreviated  on-the-job  train¬ 
ing  program  and  does  not  use  the  device  daily. 
Many  instructors  are,  therefore,  not  quali¬ 
fied  to  administer  training  in  the  device. 

The  result  is  that  training  quality,  quantity, 
standardization,  and  efficiency  suffer. 

IOS  design  is  not  just  the  physical  lay¬ 
out  of  a  work  place.  It  also  encompasses 
consideration  of  instructional  features,  in¬ 
formation  management,  software  capabilities, 
and  many  other  varied  factors.  To  properly 
incorporate  the  many  factors  into  an  effi¬ 
cient  training-effective  design  requires  a 
systematic  analysis  process.  It  is  apparent 
that  many  IOSs  currently  in  use  have  not 
evolved  through  such  an  analysis  and  design 
process. 

The  purpose  of  this  paper  is  to  summar¬ 
ize  the  problem  areas  which  have  resulted 
from  analysis  and  design  shortcomings  and  to 
present  design  principles  and  concepts  to 
overcome  these  deficiencies.  The  focus  is 
on  operational  flight  trainers  (OFT).  Many 
of  the  problems  and  recommendations,  however, 
are  applicable  to  a  wide  range  of  training 
devices. 

PROBLEM  AREAS 

There  are  significant  differences  in  the 
design  features  of  existing  simulator  IOSs. 
Some  work  better  than  others;  some  have 
sophisticated  features,  but  do  not  work  very 
well;  others  are  simple  and  work  quite  well. 
Collectively,  however,  there  are  weaknesses 
which  characterize  OFT  IOSs.  These  weaknes¬ 
ses  are  as  follows: 
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-  Layout,  labelling,  coding,  etc.  do 
not  optimize  device  operation  and 
minimize  instructor  workload. 

-  Steps  required  to  access  many  CRT 
displays  are  time  consuming  and 
inefficient. 

-  Delays  in  display  access  cause  inef¬ 
ficiencies  in  training  problem  con¬ 
trol  and  monitoring. 

-  Data  entry  methods  are  confusing, 
redundant,  and  inefficient. 

-  Many  features  and  capabilities  are 
not  needed,  not  used,  or  difficult 
to  use. 

-  Instructor  training  does  not  ade¬ 
quately  prepare  instructors  for  their 
roles  and  responsibilities  in  using 
simulators. 

-  Instructor  roles  and  responsibilities 
are  not  completely  defined  and 
implemented. 

-  Student  training  syllabi  are  poorly 
developed  and  used.  Standardization, 
organization,  efficiency,  and  qual¬ 
ity  of  instruction  are,  therefore, 
impaired. 

-  Simulator  capabilities  are  not  keyed 
to  the  training  requirements  (i.e., 
objectives)  of  the  training  system 
in  which  it  will  be  used.  The 
simulators  are,  therefore,  not  pro¬ 
perly  integrated  into  the  training 
system  as  an  efficient  element 
which  fills  a  clearly-def med  need. 

Instructor  handbooks  are  not  properly 
organized  and  formatted  to  provide 
instructor  assistance  in  operating 
the  device  in  a  training  situation. 
They  are  massive  informational 
volumes,  not  training  tools. 

When  taken  together  the  clear  impression 
is  conveyed  that  the  total  concept  of  design 
for  the  instructor  is  very  "loose"  (i.e., 
the  systematic  principles  of  man-machine 
interface,  human  factors,  and  training  tech¬ 
nology  have  not  been  properly  applied).  The 
net  result  is  that  planning  and  design  for 
operation  of  the  device  are  secondary  com¬ 
ponents  of  the  total  simulator  development 
process.  Actual  operation  of  the  device 


and  training  using  the  device,  therefore, 
suffer. 

In  many  cases  it  is  evident  that  a  num¬ 
ber  of  the  features  and  capabilities  were 
included,  not  because  they  were  clearly  re¬ 
quired  to  accomplish  the  desired  level  of 
training,  but  because  no  one  really  knew 
whether  or  not  they  should  be  there;  so  they 
were  put  in.  This  tendency  toward  overkill 
has  resulted  from  two  primary  causes:  (1) 
failure  to  perform  the  proper  training  analy¬ 
sis  efforts  leading  up  to  device  design  and 
(2)  the  explosion  in  potential  device  capa¬ 
bilities  resulting  from  technological  ad¬ 
vances  in  the  state-of-the  art.  The  former 
can  be  characterized  as  follows:  "We're  not 
completely  sure  what  we  no^d  to  teach  in  the 
device  or  how  we  should  teach  it,  so  we'll 
■"tick  these  characteri sties  in  just  to  be 
sure."  The  latter  could  go  something  like 
this:  "We  do  not  know  just  how  we'll  use 
these  instructional  features,  but  we'll  put 
them  in  anyway.  They're  neat."  The  overkill 
increases  the  requirements  for  software  and/ 
or  hardware  and  escalates  cost. 

The  easy  way  is  to  allow  simulator 
technology  to  dictate  and  include  everything 
that  it  permits  and  hope  every  contingency 
is  included.  The  rationale  sometimes  ex¬ 
pressed  "we  may  need  it  someday"  or  "its 
nice  to  have"  or  “it  doesn't  cost  anymore". 
This  approach  is  the  development  of  a  sub¬ 
stitute  aircraft  and  not  a  training  vehicle, 
with  the  result  that  due  to  tne  complexity 
of  the  IOS,  the  instructor  pilot  has  neither 
the  skill,  the  time  nor  the  inclination  to 
utilize  the  capabilities  of  the  simulator. 
Thus  training  capabilities  of  the  simulator 
are  degraded. 

While  the  capabilities  and  features  may 
be  very  desirable  on  paper,  they  tremendous !■ 
complicate  device  operations  and  increase 
costs.  Complications  result  from  the  amount 
of  information  which  must  be  handled,  the 
variety  of  data  presented  to  the  instructor, 
the  complexity  of  the  simulator  control,  t*. 
amount  of  training  required  for  lr.t'ji''  •  , 
and  human  limitations. 

assumpjj:;  :s 

The  data  on  I  OS  pm: 
set  cf  assumption-  whi  •  .  ■ 
and  a  framework  ’or 
concepts.  The  a-  ..  : 
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syllabus  will  be  developed  as  part  of  the 
overall  planning,  design,  and  fabrication  of 
a  device.  The  syllabus  will  result  from  a 
systematic  front  end  analysis  In  which  the 
simulator  Is  treated  as  a  component  of  an 
Integrated  training  system.  It  will  be  ap¬ 
proved  by  the  user  and  will  form  the  basis 
for  Implementation  of  training  on  the  com¬ 
pleted  device.  Admittedly  this  assumption 
does  not  always  embody  the  way  things  are 
usually  done.  Rather  It  embodies  the  way 
simulator  acquisitions  should  work  and  Is  a 
direction  In  which  acquisition  policies  and 
procedures  are  moving.  A  well-designed  syl¬ 
labus  is  an  important  prerequisite  to  good 
simulator  design. 

-  Training  system  and  human  engineering 
personnel  will  participate  In  the  design  of 
the  IOS.  It  is  further  assumed  that  these 
personnel  will  have  the  skills  required  to 
develop  and  validate  the  training  tasks  and 
syllabus  to  convert  the  validated  data  into 
an  efficient  IOS  station.  The  emphasis  is 
that  the  IOS  will  be  keyed  to  the  training 
requirements  and  situation,  not  to  technology, 
excess  capabilities,  and  designer  whims. 

-  Instructor  pilots  (IPs)  will  continue 
as  simulator  training  instructors  for  the 
forseeable  future.  This  assumption  has  both 
positive  and  negative  aspects.  Positively, 
it  means  that  tactically  and  with  respect  to 
flight  skills,  the  instructors  will  be  highly 
qualified.  They  will  possess  the  ability  to 
closely  Identify  with  student  problems.  Neg¬ 
atively,  they  may  not  be  highly  motivated  and 
trained  to  perform  the  roles  and  responsibil¬ 
ities  of  simulator  instructors.  IOS  design 
must,  therefore,  encompass  instructor  aid 
features  which  minimize  instructor  shortcom¬ 
ings  and  which  maximize  their  strengths. 

-  Training  of  IPs  will  not  improve  in 
the  future.  As  noted  previously,  most  In¬ 
structor  training  is  Insufficient  to  properly 
prepare  them  for  their  role  in  administering, 
monitoring,  and  evaluating  training  exercises. 
IOS  hardware  and  software  must,  therefore,  be 
easily  Interpreted  and  used,  and  must  provide 
support  to  offset  the  effect  of  Insufficient 
training. 

-  Simulator  technology  will  continue  to 
advance  In  rapid  strides.  It  will,  therefore 
be  Increasingly  Important  for  analysts  and 
designers  to  stay  abreast  of  technological 
developments  and  to  closely  evaluate  their 
applications  to  Individual  simulator  acqui¬ 
sitions. 


-  Simulators  will  be  used  as  training 
vehicles  and  not  as  substitutes  for  the  air¬ 
craft.  They  will,  therefore,  contain  only 
those  attributes  which  contribute  to  training. 
The  attributes  will  be  determined  through  a 
systematic  analysis  of  the  training  require¬ 
ments  and  the  training  system. 

DESIGN  PRINCIPLES 

The  flight  simulator  design  and  opera¬ 
tion  weaknesses,  and  the  assumptions  form 
the  basis  for  the  IOS  design  principles.  The 
design  principles  serve  two  primary  purposes: 
(1)  to  provide  a  summary  of  what  should  be 
done  to  correct  the  weaknesses  discussed 
previously  and  to  implement  the  assumptions 
and  (2)  to  provide  a  set  of  guidelines  which 
will  direct  the  development  of  IOS  design 
recommendations.  The  design  principles  are 
as  follows: 

-  Reduce  the  instructor/ operator  re¬ 
quirements  for  formal  training  by 
providing  an  instructor/operator  in¬ 
structional  aid  system  at  or  near  the 
IOS.  This  feature  will  not  only  pro¬ 
vide  training  in  the  operation  of  the 
device  but  will  actively  assist  the 
IP  throughout  the  training  exercise. 

-  Emphasize  the  use  of  automated  train¬ 
ing  as  the  normal  mode  of  training. 

-  Reduce  the  instructor/operator  work¬ 
load  by  automating  the  ancillary 
tasks. 

-  Reduce  the  number  and  type  of  data 
entry  methods  at  the  IOS. 

-  Use  "touch"  panels  on  CRTs  and  elec¬ 
tronic  "touch  pads"  on  the  IOS  con¬ 
sole  as  the  primary  data  entry 
methods. 

-  Design  IOS  layouts  to  enhance  opera¬ 
tion,  interpretation,  sequences  of 
actions,  etc. 

-  Eliminate  the  use  of  multi -function 
controls/ switches. 

-  Standardize  the  nomenclature  used  on 
controls  to  reflect  IP  terminology. 

-  Investigate  the  use  of  color  on  CRT 
displays  to  highlight  important 
points. 


-  Upgrade  student  and  instructor  train¬ 
ing  and  operatinq  materials. 

-  Improve  reliability  and  maintainabil¬ 
ity  by  use  of  touch  controls,  elimin¬ 
ating  light  pens,  etc. 

-  Reduce  cost  by  limiting  IOS  capa¬ 
bilities  to  training  requirements. 

Each  of  the  IOS  design  principles  is  dis¬ 
cussed  in  the  following  paragraphs. 

INSTRUCTIONAL  AID  SYSTEM.  To  implement  the 
principle  to  reduce  the  need  for  formal 
training  of  IPs,  it  is  proposed  that  a  com¬ 
puter  aided  instructional  (CAI)  system  be 
incorporated  as  an  integral  part  of  the  IOS. 
This  Instructional  Aid  System  (IAS)  will  have 
the  capability  of  presenting  instructional 
programs  to  the  IP  for  operation  of  the  de¬ 
vice;  provide  the  necessary  instructions  to 
the  IP  in  the  set-up,  operation,  and  conduct 
of  instructional  exercises  for  a  selected 
mode  of  operation;  and  cue  the  IP  as  neces¬ 
sary  during  a  training  exercise. 

As- a  pure  instructional  tool  it  will  be 
programmed  to  present  instruction  on  basic 
operations  of  the  device  to  include  topics 
such  as  control  locations  and  operations, 
display  formats  and  interpretation,  exercise 
control,  and  use  of  related  training  material 
(e.g.,  instructor  guides  and  checklists). 
During  set-up  and  conduct  of  exercises  it 
will  aid  the  instructors  via  prompts  which 
guide  them  through  the  required  operational 
steps.  For  example,  set-up  on  most  devices 
requires  the  instructor  to  make  a  sequence 
of  decisions  which  establish  the  character¬ 
istics  of  the  exercise.  The  IAS  would  step 
instructors  through  this  process,  thus  re¬ 
ducing  instructor  training  requirements  and 
reducing  error  rates.  During  exercises  the 
IAS  would  provide  cues  and  instructions  to 
enhance  instructor  performance,  decision 
making  etc.,  as  he  monitors,  controls,  and 
evaluates. 

AUTOMATED  OPERATIONS.  The  most  sophisticated 
simulator,  which  will  be  efficient  to  use  in 
achieving  the  training  objectives  and  which 
will  simplify  the  instructor  workload,  allow¬ 
ing  him  to  concentrate  on  Instruction  versus 
operation.  Is  the  trainer  which  Incorporates 
automated  (programed)  training  exercises. 

The  apprehension  that  automating  training 
will  be  highly  structured  and  thus  rigid  can 
be  eliminated  by  astute  planning  In  the 
development  of  a  simulator  training  syllabus 


and  the  specific  training  exercises.  By  us¬ 
ing  the  analytical  approach,  the  planning 
will  consider  all  the  variables  and  contin¬ 
gencies  required  for  achieving  the  training 
objectives  and,  therefore  provide  all  the 
required  flexibility  in  simulator  training. 

Use  of  an  automated  simulator  instruc¬ 
tional  system,  by  its  very  nature,  will  pro¬ 
mote  standardization.  The  training  exercises 
are  the  same  for  each  trainee.  The  informa¬ 
tion  presented  to  the  trainee  is  consistent 
and  in  the  correct  format  and  the  performance 
measurement  parameters  and  scoring  procedures 
are  the  same  for  each  trainee.  Evaluation 
of  the  trainee  performance  is  more  objective 
and,  therefore,  more  valid.  Due  to  simpli¬ 
city  of  the  design  of  the  IOS,  Instructor 
Pilot  activity  at  the  IOS  is  greatly  reduced, 
allowing  him  to  concentrate  on  his  instruc¬ 
tional  role. 

The  use  of  automated  instruction  will 
require  a  reorientation  in  the  concept  of 
simulator  training.  This  requires  the  recog¬ 
nition  that  the  simulator  is  a  training  ve¬ 
hicle  and  not  a  poor  substitute  for  the  air¬ 
plane.  As  a  training  vehicle  it  should  be 
responsive  only  to  the  determined  training 
requirements  as  defined  by  specific  training 
syllabi. 

The  use  of  automated  .training  exercises, 
specifically  designed  to  provide  training  in 
achieving  designated  training  objectives, 
will  eliminate  excessive  trainer  set-up  time 
required  in  a  "free  flight"  mode.  In  the 
"free  flight"  mode  of  operations  in  many 
devices,  the  instructor  must  determine,  se¬ 
lect  and  insert  every  parameter  such  as 
initial  conditions,  aircraft  location,  fly-to 
points,  ground  communications  facilities, 
geographic  displays,  environmental  factors, 
and  others.  A  manual  set-up  time  of  15-30 
minutes  is  not  unusual;  this  situation  not 
only  is  non-productive  time  for  the  trainee 
but  deprives  him  of  scheduled  training. 

It  is  recommended  that  automation  go  one 
step  further.  There  would  be  no  "free  flight' 
mode  as  currently  defined.  Free  flight  would 
be  a  cross  between  current  free  flight  and 
automation  (i.e.,  certain  operations  would 
be  automated,  and  instructor  prompting  would 
be  provided).  For  example,  the  instructor 
could  not  enter  initial  conditions  on-line. 
Rather,  he  would  always  select  from  the  pro¬ 
grammed  set  of  initial  conditions.  To  accom¬ 
modate  this  feature  the  set  of  initial  condi¬ 
tions  would  be  large  and  would  be  systemati- 


cally  derived  from  an  analysis  of  the  train¬ 
ing  requirements. 

Selection  of  Initial  conditions  to  begin 
an  exercise  would  automatically  branch  the 
IOS  to  pre-determlned  displays  and  Instruc¬ 
tions  for  Instructor  actions.  This  basic 
philosophy  of  pre-selecting  those  displays 
and  actions  which  can  be  predicted  ahead  of 
time,  based  on  the  syllabus  and  type  of 
training  to  be  conducted,  would  be  carried 
throughout  the  free  flight  mode. 

IOS  LAYOUT.  At  first  sight  the  IOS  for  a 
sophisticated  aircrew  simulator  may  be 
overwhelming  In  its  size  and  complexity.  The 
numbers  of  switches,  displays,  keyboards, 
and  other  assorted  input/output  features  may 
be  ominous  to  the  potential  simulator  In¬ 
structor.  Even  after  the  instructor  has  be¬ 
come  familiar  with  operation  of  the  device, 
there  may  still  be  a  feeling  of  being 
overwhelmed. 

The  IAS  and  emphasis  on  automation  are 
features  which  will  reduce  the  complexity 
of  the  IOS.  Even  with  these  features,  how¬ 
ever,  there  will  still  be  requirements  for 
enough  controls  and  displays  to  present  a 
confusing  operating  situation.  In  order  to 
reduce  confusion  and  Increase  the  efficiency 
of  operation,  the  IOS  design  must  be  develop¬ 
ed  through  a  systematic  human  factors  analy¬ 
sis.  The  analysis  Is  not  just  to  meet  the 
requirements  of  MIL— STD  1472C  (Human  Engi¬ 
neering  Design  Criteria  for  Military  Systems, 
Equipment,  Facilities).  It  Is  Intended  to 
yield  the  most  efficient  training/operating 
environment  possible  for  the  Instructors. 
Principles  such  as  placement  based  on  fre¬ 
quency,  criticality,  and  sequence  of  use, 
must  be  used. 

Characteristics  of  the  CRT  displays 
must  be  closely  considered.  Many  of  the  dis¬ 
plays  used  In  existing  devices  are  cluttered 
and  difficult  to  Interpret.  One  salient 
point  Is  the  possibility  of  using  color  to 
highlight  selected  portions  of  displays. 

DATA  ENTRY.  Data  entry  In  many  devices  Is 
needlessly  complicated.  For  example.  In 
some  devices  combinations  of  light  pens, 
fixed  function  keys,  and  variable  function 
keys  are  required  Interchangeably  throughout 
an  exercise.  This  mixing  of  modes  Increases 
the  time  required  to  gain  proficiency  and  the 
probability  of  error  and  confusion. 

To  remedy  this  weakness  It  Is  recommend¬ 


ed  that  data  entry  modes  be  limited  to  a  set 
of  fixed  function  electronic  touch  pads  on 
the  IOS  console  and  touch  panels  on  the  IOS 
CRTs.  The  touch  pads  available  on  each  CRT 
display  would  be  a  function  of  the  training 
situation  as  reflected  on  the  CRT  and  would 
vary  from  display  to  display.  Basically 
the  fixed  function  keys  would  provide  simu¬ 
lator  control.  The  CRT  touch  pads  would 
provide  training  exercise  control. 

STANDARDIZED  NOMENCLATURE.  Operating  inef¬ 
ficiencies  may  be  caused  by  unclear,  confus¬ 
ing,  ambiguous,  or  unfamiliar  terminology. 

To  help  remedy  the  problem,  it  is  recommended 
that  standardized  terms  be  adopted  and  used. 
Since  IPs  are  the  primary  operators  of  the 
simulators,  the  terms  used  should  be  in 
"pilotese".  Abbreviations  should  be  avoided 
when  feasible.  Coding  should  be  minimized 
and  when  used  should  be  used  in  a  clearly 
interpretable,  easily  remembered  scheme. 

SUPPORTING  MATERIALS.  Although  the  student 
and  instructor  materials  are  not  directly 
involved  In  IOS  design,  they  do  play  an  In¬ 
direct  role  In  that  they  will  be  used  In  con¬ 
junction  with  the  IOS  to  carry  out  training 
exercises.  They  should,  therefore,  be  de¬ 
signed  to  prepare  the  students  and  Instruc¬ 
tors  to  carry  out  their  roles  during  training 
exercises. 

The  IAS  Is  intended  to  take  over  some  of 
the  functions  served  by  the  instructor  train¬ 
ing  and  operating  materials.  Adjunct  written 
materials,  however,  will  still  be  required. 
These  materials  include  the  instructor  hand¬ 
book,  exercise  guides,  and  checklists.  As 
noted  previously,  the  quality  of  these  mater¬ 
ials  for  existing  simulators  is  generally 
Inadequate.  Their  shortcomings  contribute 
to  simulator  utilization  problems.  For  emer¬ 
ging  systems  they  must  be  upgraded. 

RELIABILITY  AND  MAINTAINABILITY.  Reliability 
and  maintainability  will  be  enhanced  by  the 
application  of  the  design  principles,  stated 
above,  to  the  IOS  which  In  turn  will  be  re¬ 
flected  In  the  total  simulator.  The  elimina¬ 
tion  and/or  reduction  of  switches  and  replace¬ 
ment  with  touch  controls;  elimination  of  com¬ 
plex  keyboards  and  light  pens  will  make  the 
IOS  more  reliable  and  require  less  mainte¬ 
nance. 

COST.  Reduction  In  cost  for  the  IOS/sImula- 
tor  is  a  primary  goal  In  the  application  of 
the  design  principles.  Eliminating  the 
overkill  capabilities  Inherent  in  current 


simulators  will  result  In  lower  procurement, 
operating,  and  maintenance  costs. 

DESIGN  CONCEPT 

The  design  concept  Is  based  upon  con¬ 
sideration  8f  four  data  sources: 

-  Instructor  tasks:  the  roles  and 
responsibilities  of  IPs  In  administering 
training  exercises.  For  example,  brief  stu¬ 
dent,  select  mode  of  operation.  Initialize 
exercise,  monitor  trainee,  evaluate  per¬ 
formance,  etc. 

-  Training  tasks:  tasks  the  trainee 
will  practice  In  the  device.  These  are  the 
tasks  from  a  typical  Instructional  Systems 
Development  task  listing  which  are  selected 
for  simulator  training. 

-  Design  principles  discussed  in  the 
previous  section. 

-  Technology  resulting  from  an  assess¬ 
ment  of  current  and  projected  state-of-the- 
art. 

The  design  concept  consists  of  two  com¬ 
ponents:  hardware  and  software.  The  com¬ 
ponents  interact  and  are  dependent  one  upon 
the  other.  A  major  issue  in  any  design  in¬ 
volving  both  hardware  and  software  Is  es¬ 
tablishing  the  proper  functional  balance  be¬ 
tween  the  two.  In  pointing  out  the  distinc¬ 
tion  between  hardware  and  software,  there  is 
an  emphasis  that  IOS  design  is  much  more  than 
using  good  human  factors  principles  in  deter¬ 
mining  what  the  IOS  should  look  like.  It  is 
a  systematic  process  of  determining  the  in¬ 
formation  storage,  retrieval,  display,  and 
manipulation  requirements  and  implementing 
these  requirements  in  a  combination  of  hard¬ 
ware  and  software  which  optimizes  instructor 
performance.  Major  design  concepts  are  dis¬ 
cussed  in  the  following  paragraphs. 

HARDWARE.  The  IOS  hardware  must  be  reliable, 
maintainable  and  easy  to  operate  and  yet 
must  contain  the  components  required  for 
control,  monitoring,  and  evaluation.  Major 
hardware  components  of  the  proposed  IOS  de¬ 
sign  concept  are  as  follows: 

-  The  main  console.  The  main  console 
is  compact  and  is  designed  for  a  single  in¬ 
structor/operator.  All  layout  is  consistent 
with  good  human  factors  design  principles. 
Particular  emphasis  is  on  ease  of  use  of 
controls,  orientation  of  CRTs  for  ease  of 


display  Interpretation,  and  a  functional 
work  surface  which  acconmodates  Instructor 
guides,  checklists,  etc. 

-  Instructor  Aid  Station  (IAS).  The 
IAS  is  a  separate  small  system  with  associ¬ 
ated  controls.  It  is  an  integral  part  of 
the  IOS  and  serves  two  functions:  (1)  pre¬ 
sent  instruction  on  basic  trainer  operation 
and  (2)  prompt  and  guide  instructors  during 
the  course  of  training  exercises.  It  may 
also  be  used  to  display  selected  cockpit 
instruments  (e.g.,  multi -function  display, 
horizontal  situation  indicator,  etc.). 

-  Instructor's  control  panel.  The  in¬ 
structor's  control  panel  is  the  major  "hard¬ 
wired"  part  of  the  IOS  dedicated  to  simula¬ 
tor  control.  It  consists  primarily  of  fixed- 
function  electronic  touch  pads. 

SOFTWARE.  In  order  to  combine  simplicity  of 
hardware  with  complexity  of  weapons  systems 
and  training  problems.  It  is  essential  that 
software  be  designed  to  enhance  instructor 
performance.  The  software  must  be  simple  to 
manipulate,  present  information  in  easily 
used  formats,  and  facilitate  problem  control 
and  monitoring.  Major  features  of  the  pro¬ 
posed  software  design  concept  to  meet  these 
goals  are  as  follows: 

-  Display  continuity.  A  given  display 
is  always  presented  on  the  same  CRT.  There 
is  no  switching  of  displays  between  CRTs  at 
the  Instructor's  discretion.  When  a  display 
is  selected  or  is  called  up  automatically  by 
the  software,  it  always  appears  in  the  same 
place. 

-  Standardized  displays.  Each  type  of 
display  has  a  distinct,  precisely  prescribed 
format  which  Is  always  used  for  that  type  of 
display.  Formats  are  developed  to  enhance 
Interpretation  and  use  of  the  information 
presented.  Display  highlights  are  empha¬ 
sized  by  spacing,  graphics,  bold  alphanumer- 
ics,  etc.  It  is  also  recommended  that  color 
be  used  to  highlight  portions  of  the  displays. 

-  Automatic  presentation  of  displays. 
Maximum  use  is  made  of  software  selection  of 
displays,  so  that  minimum  instructor  Inter¬ 
vention  is  required.  During  pre-programmed 
modes  all  displays  are  software  selected. 
During  free  flight  the  amount  of  instructor 
display  selection  is  minimized  through  keying 
displays  to  trainee  tasks. 

-  Minimize  steps  to  change  displays. 


104 


As  discussed  above,  display  location  Is 
standard.  There  Is,  therefore,  no  concern 
with  where  a  given  display  will  appear.  Also 
displays  which  are  candidates  to  replace  a 
given  display  which  is  being  presented  on  a 
CRT  are  selectable  using  the  touch  pads  on 
the  CRT  face.  In  most  cases  changing  dis¬ 
plays  will  require  a  one-step  touch.  In 
most  other  cases  software  design  should 
minimize  the  number  of  decision  points  (i.e., 
steps)  the  instructor  must  handle. 

-  Minimum  steps  to  manipulate  training 
exercises.  Minimum  steps  to  manipulate  is 
closely  related  to  minimum  steps  to  change 
displays.  In  this  case,  however,  rather 
than  changing  displays  for  monitoring  pur¬ 
poses  only,  the  instructor  is  locating  the 
display  from  which  he  will  affect  training 
problem  control  (e.g.,  locate  and  select  a 
specific  emergency,  locate  and  activate  a 
specific  navigational  beacon,  etc.). 

-  Large  selection  of  programed  exer¬ 
cises.  One  of  the  strengths  of  the  proposed 
approach  to  IOS  design  and  operation  is  a 
high-quality  front-end  analysis.  Among 
other  things,  the  analysis  yields  a  realistic 
training  device  syllabus  which  is  keyed  to 
the  training  requirements  and  training  situ¬ 
ation.  The  programmed  exercises  designed 
from  the  syllabus  aro,  therefore,  essential 
parts  of  the  total  training  system  and  should 
be  administered  to  each  student.  The  large 
selection  allows  consideration  of  student 
skills  and  progress,  a  variety  of  equally 
difficult  exercises  for  different  students, 
standardized  training,  progranmed  exercise 
capability  for  all  training  phases,  and 
elimination  of  time-consuming  set-up  required 
in  conventional  free  flight  exercises. 

-  Large  selection  of  initial  condi¬ 
tions.  Setting  up  initial  conditions  in  the 
free  flight  mode  on  many  simulators  is  time- 
consuming  and  may  be  non-standard.  A  large 
set  of  initial  conditions  gives  instructors 
the  flexibility  they  desire  and  Improves  use 
of  time  and  standardization.  Selection  of  a 
given  set  of  initial  conditions  would  key 
other  responses  from  the  IOS.  For  example, 
selecting  initial  conditions  for  an  aircraft 
on  the  parking  ramp  may  activate  display  of 
the  normal  cockpit  checklist,  or  selecting 
initial  conditions  for  an  aircraft  in  mar¬ 
shall  may  activate  the  carrier  approach 
display. 


teach  instructors  how  to  use  the  simulator. 

In  this  role  it  is  a  CAI  terminal  used  to 
present  information  on  the  operation  of  the 
IOS  and  on  the  procedures  for  conducting 
training  exercises.  The  second  role  is  to 
prompt  instructors  during  set-up  and  execu¬ 
tion  of  training  exercises.  In  this  role 
the  IAS  is  a  job  aid  which  improves  instruc¬ 
tor  efficiency  and  standardization  of  in¬ 
struction. 

CONCLUSION 

A  quote  from  the  2F114  (A-6  WST)  in¬ 
structor  handbook  is  as  follows:  "This  vol¬ 
ume  is  designed  to  give  the  instructor  the 
flexibility  necessary  to  provide  each  student 
with  the  complex  and  repetitive  training  he 
requires."  This  emphasis  on  flexibility  has 
led  to  IOS  designs  which  are  needlessly  com¬ 
plex  and  which  actually  hinder  training. 

This  is  not  to  say  that  flexibility 
should  be  eliminated:  certainly  not.  De¬ 
grees  of  flexibility  are  necessary  to  meet 
the  variety  of  needs  in  a  typical  FRS  train¬ 
ing  situation.  A  major  thesis  of  this  paper, 
however,  is  that  too  much  flexibility  is  a 
hinderance.  To  balance  flexibility  with 
ability  to  meet  realistic  training  require¬ 
ments,  the  IOS  should  be  accurately  program¬ 
med  based  on  the  results  of  a  thorough 
front-end  analysis.  The  software  concepts 
presented  above  provide  the  guidelines  for 
this  programming.  The  more  structured 
approach  embodied  in  the  analyses  and  result¬ 
ing  software  will  help  ensure  that  simulator 
training  is  standardized  yet  sufficiently 
flexible  to  meet  training  needs,  readily 
manageable,  and  is  easy  for  instructors  to 
learn  to  administer. 


-  IAS  teaching  and  prompting.  The  IAS 
has  two  primary  roles.  The  first  is  to 
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ABSTRACT 

One  o£  the  iaeues  facing  the  deaigner  of  sophisticated  einulatore  today 
ie  hov  to  control  their  operation.  The  traditional  approach  haa  been  to 
supply  the  inetructor/operator  with  a  series  of  hardware  devicea  (push  buttons, 
keys,  switches,  etc.)  with  which  to  command  the  machine.  A  recent  trend 
has  moved  the  functions  to  a  CRT  where  commands  are  activated  via  an  associated 
light  pen,  special  function  keys,  or,  in  some  cases  by  touching  the  CRT  screen. 
Although  the  methods  of  entering  commands  to  a  simulator  are  changing,  the 
commands  themselves  are  not. 

This  paper  explores  a  concept  in  which  series  of  lower  level  events 
or  commands  are  grouped  into  a  larger  entity  called,  for  the  sake  of  this 
discussion,  a  "task  module".  Manipulation  of  these  task  modules  permits 
simpler  operation  of  a  simulator  and  the  task  modules  themselves  provide 
natural  vehicles  for  performance  measurement,  scenario  generation,  briefing/de¬ 
briefing,  replay,  and  other  associated  functions.  In  the  future  this  concept 
can  provide  the  framework  for  modeling  intelligent  adversaries  or  supporting 
organisations  in  multi-unit  simulators. 


BACKGROUND 

S«g«gnl 

Simulators  for  flight  training  purposes 
have  been  in  use  for  many  years.  As  material 
and  energy  costs  diafe,  the  need  for  inexpensive 
effective  training  has  correspondingly  grown. 
The  simulator  is  a  natural  tool  for  this 
kind  of  training.  The  typical  simulator 
of  today  is  a  highly  mechanised  collection 
of  dials,  gauges  and  displays,  connected 
to  a  computer,  relying  on  limited  numerical 
inputs  by  a  skilled  technician  to  drive 
a  mockup  of  the  equipment  being  simulated. 
A  typical  example  is  the  2F95  F-14  Operational 
Flight  Trainer  (OFT). 

Typical  Cockpit  Simulator  Characteristics/On- 
eration 

The  F-14  Tomcat  is  a  U.8.  Navy  high 
performance  aircraft.  To  fulfill  its  role, 
the  aircraft  contains  very  sophisticated 
and  complex  avionics  and  weapon  systems. 


The  2F95  OFT  is  a  relatively  modern 
device  consisting  of  a  simulated  F-14A  cockpit 
(pilot  only)  mounted  on  a  motion  platform 
capable  of  providing  pitch,  roll,  heave, 
and  lateral  displacement  about  the  related 
axes.  Visual  scene  simulation  is  provided 
by  a  single  channel,  narrow  field  of  view 


VITAL  III-S  display  that  provides  night/dusk 
scenes  for  both  land  and  carrier-based  oper¬ 
ations. 

The  2F95  incorporates  a  single  XEROX 
Sigma  5  computer  system  for  all  simulation 
and  interaction  with  the  student  and  instructor. 
Operation  is  controlled  from  a  remote  instruc¬ 
tor/operator  station  located  away  from, 
but  in  sight  of,  the  simulated  cockpit. 


The  instructor  controls  operation 
in  two  ways,  one  sets  up  an  exercise,  and 
one  controls  operation  during  the  training 

session. 

The  set  up  operation  of  the  OFT  is 
done  by  "programming"  an  Alphanumeric  Data 
Display  (ADD)  system.  This  system  consists 
of  a  CRT  terminal  and  a  ten  key  numeric 
keypad.  The  ADD  system  has  a  total  of  22 
informational  displays,  nine  of  which  deal 
with  the  parameters  of  flight  (that  is, 
carrier  site,  see  state,  wind  state  and 
so  forth).  To  create  an  exercise,  the  infor¬ 
mation  on  these  displays  must  be  updeted 
to  reflect  the  intent  of  the  current  training 
session.  By  editing  the  information  on 
the  parameter  pages,  the  instructor  programs 
the  malfunctions  and  reposition  options 
to  be  encountered  in  the  training  exercise. 
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The  second  method  of  controlling  the 
exercise  is  via  a  special  control  panel, 
which  contains  hardware  switches,  dials, 
and  indicators  that  allow  direct  manipulation 
of  the  simulator  (fig.  1).  From  this  panel 
the  inatructor  can  reposition  the  aircraft, 
insert  malfunctions,  adjust  air  turbulence 
and  sea  state  and  control  the  motion  platform. 


Unfortunately,  this  method  of  control 
results  in  the  inatructor  spending  more 
time  playing  with  numbers  than  with  doing 
the  primary  job,  instructing  a  student. 
For  example,  to  give  a  student  experience 
in  dealing  with  an  engine  fire  on  takeoff, 
the  following  inputs  must  occur. 

1.  First,  the  plane  must  be  on  the 
end  of  the  runway  to  start  takeoff.  To 
do  this,  one  must  enter  the  ADD  page  number 
of  the  reposition  display  on  the  keypad 
mentioned  previously. 

2.  Next,  a  code  number  representing 
the  deaired  position  is  entered. 

3.  Then  a  reset  button  is  pressed. 


Nov  the  plane  is  on  the  runway. 

4.  To  program  the  malfunction,  a 
new  page  number  must  be  entered  to  put  up 
the  malfunction  display. 

5.  An  engine  fire,  represented  by 
another  code  number  must  be  added  to  the 


list  of  available  malfunctions.  Another 
number,  associating  this  malfunction  with 
a  malfunction  insertion  push  button  on  the 
control  panel,  must  be  entered. 

6.  When  the  student  starts  takeoff, 
the  instructor  monitors  the  airspeed  indicator 
and  inserts  the  malfunction  by  pushing  the 
malfunction  button  at  precisely  the  right 
time. 

7.  A  fire  light  goes  on  in  the  cockpit. 

At  no  time  during  this  button  pushing 
exercise  has  the  instructor  accomplished 
anything  that  has  a  direct  impact  on  teaching 
the  student.  In  the  worst  case,  the  entire 
instructional  function  can  be  lost  in  the 
mechanics  of  controlling  the  simulator. 


Figure  1.  OFT  Control  Panel 
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Instructor  Support  System 

To  alleviate  thia  situation  and  other 
problems  associated  with  simulator  training, 
the  U.S.  Navy  through  NAVTRAEQUIPCEN  sponsored 
the  development  an  Instructor  Support  System 
(ISS).  The  purpose  of  this  system  is  to 
take  the  mechanized  aspects  cut  of  the  training 
program  and  replace  then  with  intelligible 
user  interfaces  that  describe  an  exercise 
functionally  and  in  terms  familiar  and  acceptable 
to  any  trained  flight  instructor. 

The  ISS  completely  replaces  the  2F95 
IOS  with  a  new  station  consisting  of  two 
graphic  CRT  displays,  placed  one  above  the 
other,  with  a  touch  sensitive  device  overlaid 
on  the  lower  display.  All  information  needed 
to  conduct  a  training  session  is  presented 
dynamically  on  these  displays  and  all  control 
of  the  problem  and  the  simulator  itself 
is  via  the  selection  of  menu  choices  with 
the  touch  panel. 

The  subject  of  this  paper  is  but  one 
aspect  of  the  ISS.  Other  important  ones 
are  dealt  with  in  other  papers  presented 
at  this  workshop. 

TASK  MODULE  CONCEPT 

In  a  system  as  complex  as  the  F-14 
OFT  a  valid  training  structure  is  imperative. 
To  facilitate  such  a  structure,  logic  suggests 
the  breakdown  of  the  student  pilot's  tasks 
into  functional  groups.  For  the  sake  of 
this  discussion  these  groups  are  called 
"task  modules".  A  task  module  is  a  logically 
related  series  of  external  events  and  pilot 
actions  that  should  result  in  the  meeting 
of  predefined  criteria.  Examples  might 
be  the  takeoff  from  a  particular  airfield, 
a  pre-start  checklist,  or  a  wing  sweep  mal¬ 
function.  Task  modules  as  conceived  here 
have  certain  important  characteristics  and 
qualities.  A  brief  identification  follows, 
more  detailed  discussion  is  provided  later. 

1.  They  can  be  "run"  or  executed. 
That  is,  they  have  beginnings  and  endings 
in  time.  They  can  run  in  parallel,  that 
is,  two  or  more  task  modules  can  run  at 
the  same  time  independent  of  each  other. 

2.  They  can  be  easily  identified 
and  manipulated.  This  forms  the  basis  of 
high  level  simulator  control,  exercise  defini¬ 
tion,  replay,  and  the  like. 

3.  They  provide  the  vehicle  for 
associated  training  functions,  such  as  perfor¬ 
mance  evaluation  and  record  keeping. 

TASK  MODULE  TYPES 

In  the  development  of  this  concept 
task  modules  have  quite  naturally  fallen 


into  three  types  or  groups.  They  are  normal, 
flight  and  malfunction.  Normal  task  modules 
include  pre-  and  post-flight  checkouts  and 
checklists  during  flight.  Flight  task  modules 
encompass  those  tasks  related  to  flying 
skills,  procedures  and  navigation.  Malfunction 
task  modules  relate  to  system  failures. 

Normal 

These  are  non-emergency  procedures 
such  as  engine  starts,  checklists,  etc. 

An  example  of  this  type  is  the  takeoff 
checklist.  In  this  case,  the  task  module 
is  activated  by  an  instructor  request  at 
the  touch  panel.  At  this  time,  the  checklist 
appears  on  the  instructor's  display  containing 
the  items  to  be  accomplished  (fig.  2). 
Additionally,  the  training  system  may  monitor 
these  procedures.  The  student  pilot  will 
perform  such  actions  as  are  required,  and 
when  through,  indicates  completion  verbally. 
To  end  this  task  module,  the  instructor 
pushes  another  button.  The  checklist  disappears, 
and  data  concerning  performance  is  recorded 
for  later  replay,  review,  or  analysis. 

Flight 

Flight  task  modules  are  those  concerned 
with  the  student's  ability  to  carry  out  the 
mission,  be  it  navigation,  landing  practice, 
or  the  like.  This  can  be  exemplified  by 
the  TACAN  II  approach  to  Miramar  Naval  Air 
Station.  The  pilot  flies  a  tear  drop  pattern 
from  an  initial  approach  fix  to  a  final 
approach  fix  (fig.  3).  In  this  situation, 
the  events  beginning  and  ending  the  task 
module  are  entirely  automatic.  The  instructor 
can  devote  all  energy  to  monitoring  the 
student's  performance.  The  task  module 
starts  when  the  pilot  is  going  in  a  specific 
direction  at  a  specified  speed  and  altitude. 
When  the  student  has  started  the  approach, 
and  by  inference,  the  approach  task  module, 
monitoring  begins  again.  This  time,  however, 
the  events  being  monitored  are  those  specifically 
related  to  the  approach  task.  The  student 
flies  at  certain  speeds,  headings,  and  altitudes 
to  follow  the  proper  approach  patterns. 
When  the  student  completes  the  approach, 
based  on  speed,  altitude  and  heading,  the 
task  module  also  ends. 

Malfunction 

Malfunctions  are  activated  by  instructor 
request,  again  through  the  use  of  a  push 
button.  In  this  case,  however,  the  button 
is  on  a  touch  panel,  and  is  labeled  with 
the  name  of  the  malfunction  to  be  inserted, 
for  example,  ENGINE  FIRE.  Once  the  instructor 
has  pressed  the  button,  an  indicator  light 
appears  in  the  student  cockpit  and  a  malfunction 
checklist  appears  on  the  instructor's  display 
(fig.  4).  As  the  student  follovs  procedures, 
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Figure  4.  Malfunction  Display 


updates  appear  on  the  checklist  indicating 
activity  in  the  cockpit  as  it  occurs.  If 
a  student  completes  all  procedures  correctly, 
the  Malfunction  is  removed  automatically, 
or,  if  the  instructor  has  requested  it, 
■ay  stay  in  until  the  instructor  chooses 
to  reaove  it  annually  with  another  button 
press . 

APPLICATION  OF  THE  CONCEPT 

tfiilar or  gwtiga 

In  the  examples  above  the  relationship 
between  task  Modules  and  s  inula  tor  operation 
is  iaplied.  The  key  is  the  linking  of  the 
larger  units  of  the  training  operation  to 
the  aiaulator  directly.  In  the  ISS  this 
is  done  via  touch  selections  of  dynanic 
■enn  itens  put  on  the  display.  That  is, 
the  instructor  is  now  allowed  to  Manipulate 
the  problen  using  tools  that  are  natural 
to  the  application,  not  those  tied  to  the 
aiaulator  hardware.  In  a  sense  then,  the 
task  nodule  structure  becoaes  a  kind  of 
high  order  language,  where  pilot-relevant 
expressions  replace  numerical  code  connands. 

In  contrast  the  procedure  outlined 
earlier  for  inducing  an  engine  fire  on  takeoff, 
a  single  aenu  selection  narked  EHCIHE  FIRE 
ON  TAKEOFF  will  accomplish  the  saae  thing. 


Exercise  Definition 

Exercise  building  is  auch  simplified. 
Here  the  task  modules  can  be  likened  to 
building  blocks.  A  complete  training  session 
can  be  developed  quickly  and  conveniently 
froa  a  list  of  task  modules .  In  the  exercise 
set  up  facility  of  the  ISS,  groups  of  available 
task  modules  which  are  related  in  function 
are  presented  to  the  instructor  in  logical 
sequence  (fig.  5).  The  instructor  then 
siaply  selects  the  task  aodules  which  are 
appropriate  for  the  exercise.  All  such 
building  can  be  performed  at  the  start  of 
an  exercise,  and  thus  the  instructor  does 
not  have  to  work  "on  the  fly"  to  aake  sure 
all  exercise  components  are  covered. 

Performance  Measurement 

Performance  measurement  and  monitoring 
is  also  simplified.  Grading  parameters 
and  performance  criteria  may  be  associated 
with  specific  task  modules.  The  computer 
is  therefore  free  to  monitor  only  those 
events  which  have  direct  bearing  on  the 
task  module  procedures  being  graded.  Without 
such  a  mechanism  to  limit  the  processing 
requirements,  the  system  designer  is  faced 
with  the  prospect  of  having  to  watch  everything 
all  the  time.  This  can  be  an  overwhelming 
req  tenant . 


Ill 


ISEL  MODE  APPROACH  SELECTION 


H1SH-TACAN-/  <NKX> 


H!3H-TACAN-e  (NKX) 


RE-INIT1ALI2E  — 


Figure  5.  Exercise  Definition  Selection  Page 


A  related  benefit  in  this  area  is 
the  self  limiting  nature  of  the  performance 
data  generated.  Diagnostic  messages  and 
grades  are  only  displayed  for  task  modules 
associated  with  that  training  session. 
This  simplifies  and  reduces  the  amount  of 
data  the  instructor  needs  to  examine  to 
make  valid  decisions  rega  ng  the  student's 
performance  and  abilities. 


intelligent  adversaries  or  supporting  entities 
ia  multi-role  simulators.  For  example, 
the  skill  or  aggressiveness  of  an  enemy 
pilot  in  an  intercept  trainer  could  be  easily 
manipulated  using  the  task  module  concept. 
In  another  application  the  imaginary  crew 
of  an  ASH  aircraft  could  be  commanded  to 
perform  some  complex  manuevers  in  support 
of  a  shipboard  ASH  simulator  via  a  simple 
menu  choice. 


Debrief/Replay/Record  Keep in 


Another  area  in  which  task  modules  can  improve 
a  training  session  is  handling  mathematical 
models  of  outside  agencies.  Usually  the 
actions  of  FAA  controllers  and  other  individuals 
who  interact  with  the  pilot  can  be  linked 
to  a  single  task  module.  For  example,  all 
the  activity  of  a  GCA  controller  can  be 
linked  to  a  final  approach  task  module. 
Thua,  a  specific  function  can  be  tied  to 
a  specific  task  module.  This  contributes 
to  a  simplification  of  software  design. 
If  a  function  changes,  the  related  task 
module  can  be  altered  without  interfering 
with  programs  or  other  task  modules.  Nodularity 
is  a  natural  part  of  the  system. 

An  extension  of  this  concept,  which 
has  not  been  incorporated  into  the  188, 
is  that  task  modules  could  be  built  to  provide 


Debriefing  the  student  is  much  enhanced 
by  the  use  of  task  modules.  The  student 
can  be  shown  a  specific  task  during  a  replay 
without  extraneous  information  unrelated 
to  the  learning  experience.  Since  grading 
is  linked  to  a  task  module,  record  keeping 
becomes  more  viable  and  store  clear  to  student 
and  instructor  alike.  Any  task  module  may 
be  replayed  in  any  order,  saving  time  and 
enhancing  the  student's  understanding  by 
emphasising  the  specific  areas  that  need 
work.  This  is  in  sharp  contrast  to  the 
problems  that  ensue  when  replay  is  time 
related  and  the  action  of  interest  must 
be  searched  for,  or  worse,  when  no  replay 
ia  available. 


STANDARDIZATION  AID  FLEXIBILITY 


The  concept  of  handling  operation 
via  high  level  task  nodules  also  has  sone 
secondary  benefits. 

Generally,  the  task  nodule  also  provides 
standardization,  which  allows  student  performance 
to  be  evaluated  nore  consistently.  A  task 
module  will  always  start  at  the  sane  tine 
under  the  sane  conditions.  The  sane  procedures 
oust  be  followed  and  nothing  will  be  overlooked. 


At  first  glance  it  nay  appear  that 
this  approach  nay  reduce  general  flexibility 
in  sinulator  operation.  This  could  be  a 
problem  if  only  a  snail  nunber  of  task  nodules 
are  maintained.  However,  to  support  a  full 
syllabus  such  as  that  associated  with  F-14 
replacement  training,  a  large  nunber  (ISO 
in  the  ISS)  is  required.  This,  plus  the 
fact  that  the  task  modules  can  be  linked 
in  a  variety  of  ways,  assures  adequate  flexibi- 
bility. 

SOME  CAUTIONS 

In  order  for  this  concept  to  work 
effectively  the  task  modules  must  be  constructed 
very  carefully.  Less  than  a  very  thorough 
analysis  of  start/stop  events  can  cause 
task  modules  to  run  or  stop  running  unexpectedly. 

Haphazard  definition  of  scoring  criteria 
can  have  very  detrimental  effect  of  user 
acceptance  of  the  system. 


The  greatest  problem  faced  by  the 
developers  of  the  ISS  was  an  underestimation 

of  the  resources  required  to  define  the 
training  tasks  from  which  the  task  modules 
were  derived  and  the  time  and  effort  required 
to  create  and  checkout  the  task  modules. 
A  means  has  been  identified  to  ease  the 
latter  burden  by  automating  some  of  the 
task  module  creation  functions.  This  will 
enable  members  of  the  user's  community  to 
create  and  modify  task  modules  interactively 
at  a  computer  terminal.  This  will  not  require 
programming  skills  nor  a  knowledge  of  the 
simulator  itself,  but  it  will  require  a 
thorough  and  very  detailed  understanding 
of  the  learning  process,  not  a  trivial  task. 
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SIMPLIFIED  CONTROL  OF  THE  SIMULATOR  INSTRUCTIONAL  SYSTEM 
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INTRODUCTION. 

Recent  advances  in  information  processing  and 
display  capabilities  have  permitted  significant 
increases  In  both  the  number  and  the  complexity 
of  training  features  found  in  today's  simulator 
Instructional  systems.  These  advances  reflect 
a  combination  of  more  detailed  user  requirements 
as  well  as  the  rapidly  increasing  complexities 
of  the  systems  and  vehicles  being  simulated. 

These  overall  advances  in  capabilities  have 
produced  a  corresponding  and  steady  increase 
in  system  operating  and  control  requirements. 
Efforts  to  advance  the  state  of  the  art  have 
focused  primarily  on  system  capabilities  and 
fidelity  rather  than  instructional  value, 
often  short-changing  the  role  and  the  needs 
of  the  simulator  instructor.  These  and  other 
factors  have  contributed  to  Increasing  task 
loading  up  to  and  in  some  cases  beyond  accept¬ 
able  levels.  This  need  not  be  the  case;  in¬ 
structional  system  control  and  operation  can 


be  made  simple  and  straight-forward.  Task 
loading  can  be  dramatically  reduced  and 
instructional  effectiveness  Increased  through 
careful  front-end  activities  combined  with 
the  systematic  application  of  appropriate 
design  principles.  These  principles  concern 
the  recognition  of  the  goals  of  the  training 
program  and  the  unique  capabilities  and  re¬ 
quirements  of  the  simulator  Instructor  in 
meeting  these  goals. 

The  simulator  instructor,  like  the  flight 
instructor,  performs  several  kqy  functions 
in  the  instructional  process.  Some  of  these 
functions  can  be  automated  while  some  can  be 
facilitated  through  the  provision  and 
formatting  of  training-relevant  information. 

In  addition,  information  and  control  capa¬ 
bilities  can  be  made  available  with  minimum 
instructor  intervention.  The  instructor's 
workload  can  be  minimized  and,  equally 
significant,  the  instructor  can  be  given  ready 
access  to  the  instructional  capabilities  he 
requires. 


THE  INSTRUCTIONAL  SYSTEM. 

The  function  of  the  simulator  instructional 
system  is  to  support  the  goals  of  the  instruc¬ 
tional  program  and  to  support  the  instructor 
as  ne  performs  the  tasks  required  in  facilitat¬ 
ing  instruction.  The  instructional  system  can 
be  optimized  through  the  application  of  two  key 
design  considerations:  OPERABILITY  and  ACCEPTA¬ 
BILITY.  Operability  is  achieved  when  an  in¬ 
structional  system  is  as  easy  to  use  as  is 
practical,  while  acceptability  is  achieved  when 
the  system  makes  optimum  use  of  the  instructor's 
unique  talents  and  of  the  simulator's  unique 
capabilities.  While  these  are  somewhat  simpli¬ 
fied  and  generic  definitions  of  criteria  which 
touch  on  virtually  every  aspect  of  system  design, 
they  do  address  the  main  concerns.  For  this 
reason  they  are  the  primary  guidelines  for  the 
system  design  approach  to  be  developed  here. 

Except  for  the  aircraft  preflight,  the  jobs  of 
the  simulator  Instructor  and  the  flight  instruc¬ 
tor  are  closely  analogous.  They  perform  highly 
similar  functions,  including  the  arrangement  of 
training  conditions,  briefing,  demonstration,  per¬ 
formance  monitoring  and  diagnosis,  modification 
of  preplanned  training  exercises,  coaching  and 
guidance,  performance  evaluation  and  critique, 
and  communications  functions.  In  addition,  of 
course,  the  flight  Instructor  devotes  a  major 
part  of  his  attention  to  acting  as  a  safety 
pilot.  The  flight  Instructor  performs  these 


taski  by  observing  the  performance  of  the 
student  and  the  aircraft,  comparing  these 
performances  with  the  prescribed  performance 
standards  and  with  his  own  estimate  of  the 
capabilities  expected  of  the  student  at  a 
given  time.  In  addition,  he  facilitates  learn¬ 
ing  by  prescribing  practice  conditions  within 
the  limits  imposed  by  the  flight  environment. 

The  simulator  has  the  inherent  capability  of 
providing  more  task-relevant  information  than 
is  available  to  the  instructor  in  flight.  In 
addition,  it  has  the  inherent  capacity  for  pro¬ 
viding  and  standardizing  practice  conditions  as 
they  are  required  for  effective  instruction,  and 
it  has  the  ability  to  provide  information  in  many 
different  forms  and  relationships. 


AN  EXAMPLE. 

An  example  of  a  training  exercise  as  it  might 
occur  in  a  typical  simulator  is  used  to  illus¬ 
trate  the  application  of  twelve  basic  design 
objectives  used  to  achieve  optimum  instructional 
system  operability  and  acceptability.  These 
design  objectives  assume  that  the  simulator 
and  its  instructional  program  have  been  developed 
in  response  to  a  set  of  well-defined  training 
objectives.  The  definition  of  these  objectives 
results  in  the  identification  of  relevant  practice 
conditions,  performance  measures,  and  performance 
criteria  reflecting  student  progress  in  the 
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training  exercise.  The  design  objectives  are  as 
follows: 

1.  Minimize  the  number  of  individual  controls 
and  functions  on  the  instructor  console 

2.  Automate  non-instructional  tasks  to  the 
greatest  extent  possible 

3.  Minimize  the  instructor  control  actions 
required  to  perform  all  operating  and 
control  tasks 

4.  Minimize  requirements  for  changing  displays 
during  critical  instructional  periods; 
anticipate  and  program  displays  needed  in 
each  phase  of  the  exercise 

5.  Provide  maximum  instructor  flexibility  in 
the  control  of  the  instructional  process 

6.  Provide  continuous  trainee  performance 
feedback 

7.  Correlate  the  simulator's  graphic  display 
and  performance  data  capabilities  with 
trainee  monitoring  techniques  used  in 
the  aircraft 

8.  Minimize  requirements  to  enter  or  modify 
variable  data  through  exercise  pre-planning 

9.  Provide  instructor  selectable  performance 
data  recording 

10.  Make  maximum  use  of  relevant  advanced 
instructional  techniques  to  enhance  feed¬ 
back,  control  the  training  setting,  and 
simplify  performance  data  interpretation 

11.  Minimize  use  of  the  instructor  for  exercise 
preparation 

12.  Design  each  feature  to  minimize  requirements 
for  handbook  reference 

Many  of  these  guidelines  appear  to  be  contra¬ 
dictory.  How,  for  example,  does  one  design  an 
instructional  system  to  provide  maximum  Instructor 
flexibility  and  control  while  restricting  the 
number  of  controls,  control  actions  and  modi¬ 
fications  of  variable  data? 


The  answer  lies  In  a  thorough  analysis  of  the 
training  to  be  provided  in  the  simulator.  The 
training  analysis  must  define  the  training  ob¬ 
jectives  to  be  addressed,  the  optimum  conditions 
necessary  for  effective  training,  the  perfor¬ 
mance  parameters  to  be  monitored,  and  the  cri¬ 
teria  associated  with  acceptable  performance 
and  learning.  In  addition,  the  training 
analysis  must  anticipate  the  needs  of  both  the 
student  and  the  Instructor  In  practicing  and 
learning  and  In  directing  and  Instructing.  The 
functional  requirements  In  the  training  analysis 
are  reflected  in  the  approach  Illustrated  In 
the  following  example  as  they  indicate  antici- 


Figure  1.  Training  Management  System  Overview 


pations  of  training  events  and  instructor 
invol vement. 

The  example  used  here  assumes  the  existence 
of  a  system  by  which  academic,  simulator,  and 
flight  training  are  managed.  Figure  1  is  an 
overview  of  a  typical  training  management  system. 
Data  used  in  scheduling  training  resources  and 
aircraft  and  simulator  maintenance  as  well  as 
data  relating  to  syllabus  preparation  and  stu¬ 
dent  grading  are  handled  by  the  training 
management  system.  Data  are  entered  through 
computer  terminals  in  appropriate  areas  of  the 
training  facility. 

The  instructional  design  approach  illustrated 
in  this  example  is  concerned  with  three  major 
phases  of  a  typical  simulator  training  session. 

The  first  phase  includes  the  activities  of  the 
instructor  from  the  onset  of  his  preparation 
tasks  up  to  the  point  of  commencing  the  train¬ 
ing  session.  The  second  illustrates  instructor 
conduct  of  an  instructional  task  of  moderate 
complexity,  while  the  final  phase  covers  post¬ 
exercise  activities  from  the  completion  of  the 
final  training  task  to  the  completion  of  the 
debrief. 

Each  of  these  three  phases  begins  with  a  dis¬ 
cussion  of  instructor  activities  performed  for 
a  training  flight  in  the  aircraft,  followed  by 
the  corresponding  activities  and  supporting 
instructional  system  characteristics  required 
for  a  simulator  training  session.  A  one-to-one 
correlation  is  used  to  simplify  the  presentation. 
The  training  tasks,  task  sequences,  and  grading 
methods  and  criteria  are  representative  of  today's 
instructional  environment  and  its  requirements. 
"Single  Engine  Landings"  has  been  selected  as 
the  typical  instructional  task.  An  instructional 
environment  relating  to  a  mid-phase  aircraft 
familiarization  session  is  used  to  provide 
overall  continuity. 

TRAINING  PREPARATION  -  AIRCRAFT. 

Prior  to  commencing  the  briefing  the  instructor 
obtains  a  print-out  of  the  flight  grading  form 
by  way  of  the  remote  flight  Training  Management 
(TMS)  terminal  located  in  the  briefing  area. 

The  form  provides  an  up-to-date  list  of  the 
required  training  for  this  flight  since  it  is 
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Figure  2.  Representative  Grading  Form 

an  output  of  the  TMS  training  selection  process 
resulting  from  the  request  for  data.  The 
Instructor  then  obtains  the  complete  training 
record  file  which  Includes  all  previously  com¬ 
pleted  and  annotated  flight  grading  forms  and 
reviews  this  information  to  identify  performance 
trends,  weaknesses,  and  strengths.  External 
factors  such  as  weather,  traffic  density  and 
restrictions,  and  practice  landing  field  se¬ 
lection  are  Introduced  to  complete  the  In¬ 
structor's  overall  flight  planning  requirements. 

The  formal  briefing  Is  a  fairly  standard  in¬ 
structional  event  and  requires  no  special  con¬ 
sideration  here.  Typically,  It  includes  dis¬ 
cussions  and  review  of  the  flight  profile  or  plan, 
sequence  of  events,  environmental  factors,  per¬ 
formance  expectations,  procedural  knowledge, 
and  other  trainee  flight  preparation  require¬ 
ments,  and  other  related  areas  as  applicable. 

On  completion  of  the  briefing,  the  Instructor 


and  trainee  proceed  to  the  flight  line,  review 
the  assigned  aircraft  maintenance  data  by  way 
of  the  remote  aircraft  maintenance  TMS  terminal, 
and  then  perform  the  aircraft  preflight  con¬ 
cluding  the  preparations  for  flight. 


TRAINING  PREPARATION  -  SIMULATOR. 


Instructor  preparations  for  the  corresponding 
(mid-phase  familiarization)  simulator  session 
are  identical  to  those  required  for  the  aircraft 
excepting  those  areas  that  pertain  solely  to  the 
actual  flight  (i.e.,  external  environmental 
factors,  air  traffic  instructions,  aircraft  pre¬ 
flight).  The  grading  form  is  obtained  by  the 
instructor  through  the  remote  simulation  TMS 
terminal  located  in  the  simulator  briefing  area. 
The  format,  identical  for  both  flight  and  simu¬ 
lator  applications,  is  shown  in  Figure  2. 


As  an  output  of  the  TMS  the  form  provides 
the  instructor  with  supporting  information 
that  is  not  available  through  conventional 
means.  Note,  for  example,  that  a  performance 
indicator  (o)  is  identified  for  each  of  the 
listed  training  areas  or  tasks  preceded  by  the 
letters  P  or  R.  This  indicator  represents  a 
past  performance  average  accumulated  from  all 
previous  instruction.  The  letter  P  denotes 
tasks  or  performance  areas  primarily  introduced 
and  graded  while  R  identifies  areas  of  difficulty 
where  performance  has  been  substandard  and  addi¬ 
tional  (beyond  the  norm)  review  and  instruction 
is  required.  Tasks  preceded  by  the  letter  I  are 
to  be  introduced  for  the  first  time,  while  A 
identifies  areas  of  advance  instruction  to  be 
introduced  providing  time  and  trainee  performance 
permit. 


Upon  completion  of  the  briefing,  the  instructor 
and  trainee  proceed  to  the  simulator,  pausing 
briefly  at  the  remote  simulator  maintenance  TMS 
terminal  to  review  simulator  and  instructional 
system  status.  Upon  arrival  at  the  instructor 
console  the  START  function  is  selected  on  the 
instructional  system  input  device  configured 
as  shown  in  Figure  3. 
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Figure  3.  Instructor  Input  Device 
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This  action  Initializes  all  resources  of  the 
Instructional  system  In  preparation  for  the 
activation  of  the  training  selection  process. 

To  simplify  the  instructor's  Input  procedures 
all  displays  and  repeaters  are  blanked  and  the 
following  Instruction  Is  provided. 

ENTER  TRAINEE  IDENTIFIER 

The  four  digits  selected  on  the  numeric  pad 
(Figure  3,  extreme  right)  are  repeated  for 
verification  on  the  CRT  Immediately  beneath 
the  displayed  Instruction.  Assuming  the 
selection  Is  acceptable  to  the  TMS  (incorrect 
or  unacceptable  entries  are  Ignored  and  specific 
correction  instructions  provided)  ENTER  causes 
the  displayed  data  to  be  cleared  and  the  following 
data  and  instruction  presented: 

E  FAMILIARIZATION 
m  INSTRUMENTS 
GO  NAVIGATION 
S3  FORMATION 
nn  TACTICS 
[6J  CARRIER  QUAL 

SELECT  PHASE 

The  phase  is  identified  through  selection  of 
the  corresponding  enclosed  numeric,  in  this 
case  1,  for  the  familiarization  phase  in  the 
instructor's  input  device  (Figure  3,  lower 
left).  Selection  again  causes  all  display 
data  to  be  cleared,  and  through  direct  inter¬ 
face  with  TMS,  initiates  the  training  selection 
process.  After  a  short  pause  (during  which  an  IN¬ 
PROGRESS  message  Is  provided),  the  following  in¬ 
struction  and  corresponding  control  function  are 
displayed: 

E  INITIALIZE  FOR  TRAINING 

Selection  causes  a  complete  training  initiall 
zatlon  to  occur  by  positioning  and  configuring 
the  simulated  aircraft,  selecting  and  depicting 
the  appropriate  visual  scene  content  to  both  the 
Instructor  and  trainee,  and  displaying  the  appli¬ 
cable  monitoring  and  control  data  for  the  initial 
task  at  the  instructor  console.  Upon  completion  of 
these  processes,  the  simulator  is  placed  in  freeze 
and  a  READY  message  is  displayed  to  the  instructor. 

In  review,  the  simulator  preparation  procedures 
have  Included: 

1.  Selection  of  the  START  function  to  perform 
a  complete  instructional  system  reset. 

2.  ENTERing  the  trainee  four-digit  Identifier. 

3.  Selecting  the  desired  Instructional  phase. 

4.  Activating  the  training  intlallzatlon  process. 

These  procedures  differ  significantly  from 
those  used  with  systems  that  employ  extensive 
and  time-consuming  selection  and  modification 
of  Instructional  data  In  the  special  exercise 


preparation  or  planning  mode.  Many  of  these 
systems  extend  simulator  turnaround  times  beyond 
acceptable  limits  and  often  require  extensive 
Instructor  knowledge  of  complex  and  cumbersome 
data  manipulation  procedures. 

INSTRUCTIONAL  TASK-AIRCRAFT. 

The  flight  has  logically  and  sequentially  pro¬ 
gressed  on  through  to  the  pattern  entry,  and  the 
practice  of  previously  introduced  normal  landings 
has  been  concluded.  As  an  Introduction  to  the  next 
maneuver  the  instructor  takes  control  of  the  air¬ 
craft  and  demonstrates  a  single-engine  down¬ 
wind  and  approach  to  a  touch-and-go  landing 
(simulated  by  retarding  of  throttle  to  idle). 
Extensive  verbal  commentary  is  provided  on 
cockpit  procedures  and  aircraft  handling  tech¬ 
niques  Including  the  shut-down  procedures  for 
simulated  engine  fire  on  down-wind.  In  addition, 
the  instructor  must  initiate  and  respond  to 
required  voice  communications  with  the  tower 
and  monitor  the  traffic  pattern  to  insure  safe 
aircraft  separation  at  all  times. 

Upon  completion  of  the  demonstration  the  in¬ 
structor  returns  the  aircraft  to  its  normal 
two-engine  pattern  configuration,  and  once  re¬ 
established  on  downwind,  control  is  returned 
to  the  trainee. 

The  instructor  then  introduces  the  maneuver  by 
announcing  a  simulated  engine  fire  and  begins 
to  monitor  the  trainee's  procedural  response  and 
control  of  the  aircraft,  coaching  and  guiding  as 
necessary.  In  this  capacity  the  instructor's 
sequence  of  monitoring  activities  might  typically 
include: 

1.  Procedural  response  to  the  simulated  engine 
fire  as  related  verbally  by  the  trainee; 
the  trainee  reduces  the  throttle  to  idle 

at  the  appropriate  time  to  simulate  the 
engine  shutdown 

2.  Response  of  the  trainee  to  the  aircraft 
flight  requirements  (e.g.,  the  addition  of 
power  on  the  remaining  engine  to  maintain 
altitude  and  airspeed,  maintaining  balanced 
flight,  smooth  basic  airwork,  etc.) 

3.  Establishing  the  proper  downwind  path  for 
the  existing  wind  condition  and  arriving 
at  an  optimum  single  engine  abeam  distance 

4.  Simulated  transmission  to  the  tower  requesting 
emergency  clearance 

5.  Completion  of  landing  checks  and  trans¬ 
mission  to  the  tower  requesting  landing 
clearance  insures  gear  check  is  verified 
and  landing  clearance  is  received  and 
acknowledged 

6.  Basic  aircraft  control  during  the  approach, 
focusing  on  angle  of  bank,  rate  of  descent, 
angle  of  attack,  power  setting,  verifies 
parameters  at  key  checkpoints 
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7.  Final  approach  for  proper  distance  at  roll¬ 
out,  smooth  basic  alrwork,  proper  alignment, 
minor  corrective  adjustments 

8.  Touchdown  In  fl.st  third  of  the  runway  on 
center  line  at  the  optimum  rate  of  descent 
and  angle  of  attack 

9.  Acceleration  on  the  runway  for  aircraft 
control  on  center  line,  and  smooth  power 
application 

10.  Optimum  pitch  angle  at  lift-off;  stable 
lateral  control,  smooth  acceleration  to 
single-engine  climb  speed,  and  a  positive 
rate  of  climb  throughout  to  pattern 
altitude 

11.  Proper  angle  of  bank  during  downwind  turn 
to  establish  an  optimum 

distance  from  the  runway 

In  addition,  and  throughout  the  maneuver.  In¬ 
struction  Is  provided,  the  traffic  pattern  Is 
monitored  for  safe  separation,  all  transmissions 
on  the  selected  frequency  are  monitored,  and 
trainee  performance  Is  noted  for  post-flight 
analysis,  debriefing,  and  grading.  The  maneuver 
is  repeated  as  required  to  achieve  the  desired 
level  of  skills  development  for  this  flight. 

INSTRUCTIONAL  TASK-SIMULATOR. 

The  corresponding  simulator  training  session 
has  also  logically  and  sequentially  progressed 
on  through  all  the  pattern  entry,  and  the  prac¬ 
tice  of  previously  Introduced  normal  landings 
has  been  concluded.  Upon  instructor  request, 
the  Instructional  system  Is  advanced  to  the 


next  task  area  (SINGLE-ENGINE  LANDINGS)  as  the 
trainee  completes  the  crosswind  turn  to  the 
downwind  leg.  Through  this  simple  request 
(discussed  later)  the  required  Instructional 
system  features  (visual  repeater,  CRT  displays, 
and  supporting  programs)  are  selected  and 
configured  to  monitor  and  control  this  specific 
training  event  as  It  evolves  (In  addition  to  a 
view  of  the  visual  scene  of  Interest,  the 
Instructor  Is  provided  with  two  CRT  displays 
configured  as  shown  In  Figure  4). 

The  primary  display  (on  the  left)  provides  a 
graphic  depiction  and  tabular  performance  list¬ 
ing  In  the  main  areas,  with  supporting  data  and 
controls  to  the  right.  The  secondary  CRT  display 
(on  the  right)  provides  the  Instructor  with  a 
view  of  the  applicable  cockpit  Instruments  and 
Indicators  with  supporting  problem  and  task 
controls  depicted  to  the  right.  The  arrangement 
of  the  Instruments  and  Indicators  corresponds 
to  the  rear  aircraft  cockpit  configuration  to 
support  the  normal  In-flight  scan  patterns  of 
the  Instructor. 

Figure  4  also  Identifies  the  applicable  control 
functions  from  the  Instructor  Input  device 
(Figure  3)  used  to  Interact  with  each  display 
type.  Since  the  functional  arrangements  are 
dedicated,  the  requirement  to  assign  controls 
to  a  CRT  and  the  accompanying  Input  errors  that 
often  result  from  such  an  arrangement  have 
been  eliminated. 

Continuing  with  the  task,  the  Instructor  takes 
control  of  the  simulated  aircraft  by  activating 
freeze  (Figure  3,  upper  right)  and  alerts  the 
trainee  to  an  impending  Initialization.  As  an 
Introduction  to  the  maneuver  the  Instructor 
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selects  function  6  by  wqy  of  the  Input  device 
(Figure  4)  to  access  the  prerecorded  demonstra¬ 
tion.  Wten  selected,  the  supporting  data 
area  of  the  primary  display  Is  updated  as  shown 
In  Figure  5.  Function  1  Is  selected,  causing 
the  simulated  aircraft  to  INITIALIZE  for  the 
start  of  the  demonstration.  Upon  completion, 
a  READY  Indication  Is  provided  (lower  area) 
with  the  system  awaiting  the  release  of  freeze 
to  Initiate  the  playback.  Function  2  Is  selected 
to  activate  the  accompanying  prerecorded  verbal 
commentary. 

Upon  freeze  release,  the  Instructional  system 
takes  control  of  the  simulated  aircraft  and 
demonstrates  a  prerecorded  single-engine  down¬ 
wind  approach  to  a  touch-and-go  landing  (also 
simulated  by  retarding  the  throttle  to  idle). 

The  simulated  single-engine  configuration  Is 
again  used  to  maintain  continuity  with  the  pre¬ 
viously  described  aircraft  training  flight 
maneuvers.  Extensive  prerecorded  verbal  com¬ 
mentary  is  provided  on  cockpit  procedures  and 
aircraft  handling  techniques,  including  the 
shutdown  procedures  for  a  single-engine  fire 
on  down-wind.  Communications  with  the  tower 
are  included  in  the  commentary. 

As  the  simulated  aircraft  passes  the  90-degree 
position  in  the  approach,  the  instructor  momen¬ 
tarily  deactivates  the  verbal  commentary  to  pro¬ 
vide  additional  remarks  that  are  applicable  to 
this  specific  trainee  based  on  past  performance 
trends.  At  touchdown  the  commentary  is  reacti¬ 
vated. 

Throughout  the  demonstrated  maneuver,  the  CRT 
displays  and  visual  repeater  provide  all  normal 
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Figure  5.  Demonstration  Control 


feedback  to  the  instructor,  including  the  per¬ 
formance  data  (discussed  later)  depicted  in  the 
lower  main  area  of  the  primary  display  (Figure  4). 

Once  the  simulated  aircraft  returns  to  the  down¬ 
wind  leg  and  while  still  under  demonstration 
control,  a  normal  flight  configuration  is  re¬ 
stored  and  the  primary  display  is  cleared  of 
all  accumulated  track  and  performance  history. 

At  this  point,  the  instructor  alerts  the  trainee 
and  returns  control  of  the  simulated  aircraft 
to  him  (Figure  5  -  Function  5)  and  then  resets 
the  supporting  data  area  to  its  original  data 
configuration  (Function  6).  Once  assured  that 
the  trainee  is  in  comfortable  control  of  the 
situation,  the  instructor  introduces  the  maneuver 
by  announcing  a  simulated  engine  fire  and  begins 
to  monitor  the  trainee's  procedural  response 
and  control  of  the  simulated  aircraft,  coaching 
and  guiding  as  necessary.  In  this  capacity,  the 
instructor's  sequence  of  monitoring  activities 
and  supporting  system  control  inputs  might 
typically  include: 

1.  Procedural  response  to  simulated  engine 
fire  as  related  verbally  by  the  trainee 
through  the  simulator  communications  system. 

The  instructor  verifies,  the  the  correct 
throttle  is  reduced  to  idle  at  the  appropriate 
time  by  way  of  the  ACTION  MONITOR  feature 
(primary  display)  or  the  THROTTLE  position 
status  (secondary  display).  No  control  or 
selection  inputs  are  required. 

2.  Response  of  the  trainee  to  the  immediate 
aircraft  flight  requirements  (e.g.,  the 
addition  of  power  on  the  remaining  engine 
to  maintain  altitude  and  airspeed,  main¬ 
taining  balanced  flight,  smooth  basic  air- 
work,  etc.)  by  scanning  the  cockpit  instru¬ 
ments  and  indicators  by  way  of  the  secondary 
display  in  the  forward  view  scene  of  Interest 

in  the  visual  repeater.  No  control  or  selection 
inputs  are  required. 

3.  Establishing  the  proper  downwind 
for  the  existing  wind  conditions  and 
arriving  at  an  optimum  single-engine  abeam 
position  by  observing  the  aircraft  symbol 
and  track  history  depicted  relative  to  the 
optimum  downwind  corridor  (primary  display, 
upper  area)  and  abeam  position  (key  check¬ 
point  B)  and  by  scanning  the  abeam  view 
area  of  interest  on  the  visual  repeater. 

No  controller  selection  inputs  are  re¬ 
quired. 

4.  Simulated  trainee  transmissions  to  tower 
requesting  emergency  clearance  with  the 
simulated  communications  system.  Frequency 
Is  verified  on  the  ACTION  MONITOR  feature 
(primary  display)  and  the  Instructor  re¬ 
sponds  to  the  transmission  or  selects  a 
pre-recorded  message.  Available  messages 
are  accessed  on  the  PROBLEM  CONTROL  INDEX 
(secondary  display.  Function  11).  The 
desired  message  (Figure  6)  is  activated 
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Figure  6.  Message  Control 

upon  selection  of  the  appropriate  function 
—  In  this  case,  12. 

Completion  of  landing  checks  by  observing 
the  ACTION  MONITOR  feature  and  the  cockpit 
Instruments  and  Indicators  display.  Trans¬ 
mission  to  the  tower  requesting  landing 
clearance  Is  monitored  In  the  simulator 
communications  system  and  the  Instructor 
transmits  or  activates  landing  clearance 
(Figure  6)  and  monitors  for  acknowledgment 
and  gear  .check  verification  from  the 
trainee. 

Basic  aircraft  control  during  the  approach 
focusing  on  angle  of  bank,  rate  of  descent, 
angle  of  attack,  and  power  setting  as  pro¬ 
vided  by  the  cockpit  Instruments  and  Indi¬ 
cators  (secondary  display);  verifies  parame¬ 
ters  at  key  checkpoints  by  observing  the 
aircraft  track  history  and  performance  data 
areas  of  the  primary  display  reinforced  by 
the  varying  scene-of-lnterest  view  from  the 
visual  repeater.  No  control  or  selection 
Inputs  are  required. 

Final  approach  for  proper  distance  and 
runwey  alignment  at  roll-out  by  observing 
the  aircraft  symbol  relative  to  the  Ideal 
distance  (checkpoint  G)  and  the  optimum 
path  reinforced  by  the  visual  scene  of 
Interest.  Basic  alrwork  and  minor  correc¬ 
tive  techniques  are  observed  In  the  cockpit 
Instruments  and  Indicators  display  and  are 
also  reinforced  by  the  view  of  the  visual 


scene  of  Interest;  no  control  or  selection 
Inputs  are  required. 

Touchdown  on  the  first  third  of  the  runway 
as  indicated  on  the  runway  depiction  (pri¬ 
mary  display)  on  center  line  as  viewed  from 
the  visual  scene  of  interest  and  at  the  op¬ 
timum  attitude  rate  of  descent,  etc.  as 
depicted  In  the  performance  area  of  the 
primary  display  (checkpoint  TD),  no  control 
or  selection  inputs  are  required. 

Acceleration  on  the  runway  for  center-line 
control  as  monitored  in  the  visual  scene 
of  interest  and  smooth  power  application 
depicted  in  the  THROTTLES  status  area  of 
the  secondary  display;  no  control  or 
selection  inputs  are  required. 

Optimum  lift-off  attitude  (performance 
data  area,  checkpoint  LO),  smooth  acceler¬ 
ation  to  single-engine  climb  speed,  and  a 
positive  rate  of  climb  to  level  off  at 
pattern  altitude  as  observed  on  the 
cockpit  instruments  and  reinforced  in 
the  visual  scene  of  Interest.  The  In¬ 
structor  transmits  the  downwind  clear¬ 
ance  or  activates  the  applicable  pre¬ 
recorded  message. 

Adjustments  to  angle  of  bank  during  the 
downwind  turn  to  arrive  at  an  optimum 
distance  abeam  the  runway  as  depicted  by 
the  aircraft  track  history  on  the  primary 
display,  supported  by  the  cockpit  Instru¬ 
ments  and  the  visual  scene  of  Interest  for 
basic  aircraft  control  and  alrwork.  No 
control  or  selection  inputs  are  required. 


Upon  completion  the  primary  data  display  pro¬ 
vides  the  instructor  with  a  concise,  graphic 
and  alphanumeric  performance  history  of  the 
maneuver.  When  combined  with  occasional  notes 
on  basic  alrwork  and  general  performance  trends 
recorded  as  required  by  the  Instructor,  a  com¬ 
prehensive  base  of  Information  Is  made  available 
for  post-training  analysis  debrief  and  perfor¬ 
mance  reassessment.  A  sample  primary  data 
display  depicting  a  completed  touch-and-go 
Is  shown  In  Figure  7.  (Same  of  the  depicted 
data  is  applicable  to  additional  Instructional 
capabilities  to  be  addressed  shortly.) 

The  HARDCOPY  feature  (Figure  4,  Function  2) 
provides  a  printed  copy  of  the  entire  display 
as  It  appears  on  the  CRT  and  1$  commanded  by 
the  Instructor  at  the  conclusion  of  each 
maneuver.  The  print-out  (which  does  not  affect 
the  operability  of  the  system)  occurs  when  se¬ 
lected,  thus  avoiding  storage  problems  and  the 
delays  often  encountered  when  processing  large 
amounts  of  accumulated  data  at  the  conclusion 
of  training.  Once  a  print-out  has  been  commanded, 
the  track  history  and  performance  data  nuty  be 
cleared  (Figure  4,  Functions  3  and  4)  at  the 
Instructor's  option  In  preparation  for  the  next 
maneuver. 
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Figure  7.  Sample  Primary  Display  Data 


Note  that  a  number  of  additional  simulation 
capabilities  are  provided  to  the  instructor; 
these  offer  training  assistance  and  task  control 
not  available  during  aircraft  instruction. 

The  supporting  data  area  of  the  secondary  dis¬ 
play  provides  access  to  a  variety  of  these  man¬ 
euver-applicable  control  features  (Figure  4, 
PROBLEM  CONTROL  INDEX).  In  addition  to  the 
prerecorded  messages  discussed  previously, 
environmental  controls  and  engine  failures 
are  also  available  for  selection. 

The  index  Includes  control  of  environmental 
conditions  and  parameters  that  are  required 
in  providing  the  flexibility  for  complete 
instruction.  Changing  the  wind  component, 
for  example,  requires  the  trainee  to  alter 
his  approach  path  over  the  ground,  while 
varying  the  temperature  affects  engine  perfor¬ 
mance  and  resultant  power  requirements.  In¬ 
creasing  the  turbulence  level  makes  smooth  basic 
airwork  more  difficult,  while  limiting  visi¬ 
bility  effects  judgments  which  depend  on  the 
pilot's  perception  of  the  flight  area.  Varying 
aircraft  weight  and  center  of  gravity  can 
totally  change  handling  characteristics  requir¬ 
ing  altogether  different  approach  techniques. 
Since  the  simulator  provides  the  only  safe 
environment  for  the  development  of  proficiency 
in  single-engine  landings,  the  Index  also  pro¬ 
vides  access  to  selected  failures  to  realis¬ 
tically  create  the  required  conditions. 

All  selection  and  control  functions  are  accom¬ 
plished  in  the  supporting  data  area  allowing 
the  cockpit  Instruments  and  indicators  to  be 
continuously  displayed  for  performance  In  status 
monitoring.  All  control  actions  (e.g.,  selection 
of  desired  malfunction,  wind  component,  air¬ 
craft  weight,  etc.)  are  accomplished  through 
use  of  the  enclosed  numeric  functions  (Ref¬ 
erence  7,  Message  Activation).  The  selected 


conditions  are  displayed  and  the  lower  graphic 
area  of  the  primary  display  as  shown  In  Figure 
7.  Active  malfunction  titles  are  depicted  In 
the  lower  left  and  the  letter  M  Is  displayed 
In  the  graphic  maneuver  area  at  the  point  of 
activation.  This  Information  provides  the 
Instructor  with  a  continuous  display  of  simu¬ 
lator  status  during  training  and  also  facili¬ 
tates  post-training  maneuver  reconstruction 
and  debriefing. 

The  INITIALIZED  function,  also  found  In  the 
supporting  data  area  immediately  above  the 
PROBLEM  CONTROL  INDEX,  resets  the  simulated 
aircraft  to  an  optimum  position  from  which 
the  task  may  be  commenced.  As  a  visual 
aid  to  the  instructor,  the  position  is 
identified  In  a  graphic  area  (Figure  7)  by 
depicting  the  corresponding  function.  Control 
and  system  checks  are  performed  upon  selection 
and  appropriate  cuing  messages  provided  to 
Insure  the  simulated  aircraft  is  compatible 
with  the  demanded  configuration  prior  to  re¬ 
lease  of  control  to  the  trainee. 

The  ACTION  MONITOR  system  depicted  in  the 
supporting  data  area  of  the  primary  display 
(Figure  4)  Is  a  continuously  active  procedure 
monitoring  display  feature  that  lists  all 
trainee  actions  that  are  detectable  by  the 
computational  system.  The  CLEAR  MONITOR 
function  provides  the  Instructor  with  a  means 
of  erasing  all  displayed  actions  in  preparing 
to  monitor  a  specific  procedure  or  sequence. 

As  a  part  of  the  primary  display,  ACTION 
MONITOR  data  is  included  in  all  HARDCOPY  print¬ 
out  selections. 

interaction  with  the  final  instructional  system 
feature  assigned  for  use  with  this  task  and  to  be 
addressed  here  is  also  accomplished  with  the 
primary  display.  Selection  of  the  depicted 
function  (Figure  4)  reconfigures  the  supporting 
data  area  with  typical  RESET  and  REPLAY  control 
features  as  shown  in  Figure  8.  As  a  visual  aid 
to  the  instructor  in  selecting  the  optimum  time 
increment,  positions  corresponding  to  the  available 
reset  points  are  depicted  in  the  graphic  area  of 
the  primary  display  on  the  simulated  aircraft 
track  history  (Figure  7,  Diamond  Symbols). 

These  basic  and  straightforward  control  and 
monitoring  features  offer  the  instructor  all 
of  the  flexibility  required  to  successfully 
complete  the  prescribed  Instruction  with  mini¬ 
mum  distraction  and  maximum  commonality  with 
aircraft  training  procedures. 

Obviously,  the  simulator  provides  much  greater 
capability  for  the  control  of  training  in  many 
significant  respects  than  the  aircraft.  By  the 
same  token,  however,  these  capabilities  must  be 
implemented  in  the  simulator  and  In  the  Instruc¬ 
tional  system  with  operability  and  acceptability 
as  primary  guidelines.  These  are  reflected  In 
the  example  used  here  as  unique  simulator  capa¬ 
bilities  are  organized  to  support  the  methods 


122 


RESET/REPLAY 


SELECT  TIME -MINS 

□  0  0  0  0 


SELECT  9  FOR  RESET 
ONLY 


fi~]  AUDIO  OFF 


01  SLOW  TIME  OFF 

liTI  TERMINATE  AT 
—  PRESENT  POSITION  IN 
FREEZE 


FI  RELEASE  TO  TRAINEE 
CONTROL 


|10|  RESET  DISPLAY 


READY  -  RELEASE  FRZ 


Figure  8.  Typical  Reset  and  Replay  Controls 


of  Instruction  with  which  the  flight  Instructor 
Is  familiar. 


Returning  to  the  parallel  scenarios  —  the 
Instructor  has  to  his  satisfaction  completed 
a  desired  amount  of  training  In  single-engine 
landings.  The  NEXT  TASK  Is  accessed  through 
selection  of  the  depicted  function  (Figure  4) 
causing  the  Instructor  display  system  to  update 
accordingly. 

POST-TRAINING  ACTIVITIES  -  AIRCRAFT. 

Upon  completion  of  the  prescribed  secure  pro¬ 
cedures  a  post-flight  aircraft  Inspection  Is 
performed,  the  appropriate  maintenance  and  flight 
record  completed  (applicable  data  Is  entered 
Into  the  remote  aircraft  maintenance  TMS 
terminal),  and  the  Instructor  and  trainee 
return  to  the  briefing  facility. 

The  flight  Is  then  reconstructed  for  analysis 
and  debrief  using  the  partially  completed  grad¬ 
ing  form  (without  final  grade  adjustments  and 
comments),  the  performance  notes  recorded 
throughout  the  flight  and  the  Instructor's 
ability  to  recall  events  and  situations. 

Maneuvers  requiring  extensive  discussion 
are  normally  diagramed  on  the  chalkboard 
allowing  the  Instructor  to  graphically 
depict  errors.  The  diagram  Is  then  annotated 
to  Identify  points  where  correction  and  Im¬ 
provement  are  required.  With  respect  to  the 
single-engine  landing  task,  the  Instructor  must 
be  prepared  to  address  trainee  performance 


where  necessary  In  each  of  the  11  previously 
Identified  monitoring  areas  for  each  maneuver 
completed.  The  success  of  the  debrief  depends 
entirely  on  his  ability  to  accurately  recreate 
the  scene.  Identify  the  areas,  and  then  provide 
the  trainee  with  constructive  remedial  Information 
for  subsequent  application. 

The  next  activity  In  the  debrief  Is  the  com¬ 
pletion  of  the  grading  form  and  a  review  of 
Its  contents  with  the  trainee  (the  Ingredients 
of  a  successful  debrief  also  tend  to  produce 
valid  performance  assessment).  The  applicable 
information  Is  then  entered  Into  the  remote 
flight  TMS  terminal  and  Integrated  with  the 
existing  data  in  the  trainee  history 
file.  The  training  selection  process  Is  then 
activated  (as  previously  described).  The  con¬ 
tents  of  the  next  aircraft  familiarization  train¬ 
ing  flight  extracted  on  the  grading  form  Is 
printed  out  at  the  remote  terminal.  The  con¬ 
tents  of  the  form  and  the  corresponding  flight 
preparation  requirements  are  briefly  reviewed 
with  the  trainee  (who  then  retains  the  form 
for  subsequent  reference).  The  Instructional 
event  is  concluded  when  the  completed  grading 
form  Is  placed  in  the  trainee's  folder  to 
support  subsequent  briefing  and  Instructional 
activities. 

POST-TRAINING  ACTIVITIES  -  SIMULATOR. 

Upon  completion  of  the  prescribed  secure  pro¬ 
cedures,  the  simulator  status  is  accessed  and 
then  the  instructor  and  trainee  return  to  the 
briefing  facility  (pausing  at  the  remote  simulator 
maintenance  TMS  terminal  to  enter  applicable 
status  information). 

The  training  session  is  then  reconstructed  for 
analysis  and  debrief  using  the  partially  completed 
grading  form.  Instructor-selected  print-outs  and 
primary  display  data,  performance  notes  recorded 
throughout  the  training  session  and  the  In¬ 
structor's  recollection  of  key  events  and  situations 

The  grading  form  now  Includes  the  performance 
for  tasks  as  assigned  during  the  course  of  the 
training  session  as  shown  In  Figure  9.  When 
combined  with  the  past-performance  Indicators 
(o)  a  correlation  between  past  and  present 
trends  Is  provided  to  reinforce  overall  per¬ 
formance  assessment. 

Maneuver  reconstruction  Is  significantly  en¬ 
hanced  and  reinforced  with  the  addition  of 
the  primary  display  print-out.  In  the  case 
of  the  single-engine  landing  task  this  print¬ 
out  provides  the  Instructor  with  virtually 
all  of  the  Information  required  to  assess  the 
eleven  previously  Identified  performance  moni¬ 
toring  areas.  In  addition,  these  print-outs 
annotated  during  the  course  of  the  debrief  may 
be  retained  by  the  trainee  for  subsequent  reference. 

After  completion  of  the  grading  form  and  the 
review  of  its  contents  with  the  trainee,  the 
applicable  Information  Is  entered  In  the  remote 


123 


PLUMBER,  U.R. 

ENS 

1234  | 

SCREAMER,  I.M. 

LT 

5678 

FAM  SESSION  4 

08/10/82 

PLANNED  130 

ACTUAL  1:28 

_ 

AA 

A 

BA 

u 

P 

HEADWORK 

o 

P 

PROCEDURES 

o 

P 

BASIC  AIRWORK 

o 

P 

START  MALFUNCTIONS 

X 

P 

TAXI  MALFUNCTIONS 

j_5: 

X 

1 

ABORTED  TAKEOFF 

X 

~p1 

NORMAL  TAKEOFF 

X 

p 

CLIMBOUT/DEPARTURE 

X 

R 

BREAK  TURN  STALL 

X 

[_2J 

1 

ENGINE  FIRE 

X 

A 

COMPLETE  HYD  FAILURE 

X 

i 

EMERGENCY  DESCENT 

X 

P 

VFR  RECOVERY 

X 

p 

PATTERN 

(T1 2 3 4 5 

p 

NORMAL  LANDINGS 

X 

i _ 

1 

SINGLE  ENGINE  LANDINGS 

XI 

A 

HYDRAULICS  OFF  LANDINGS 

~P 

SHUTDOWN 

"IT1 

_ 

Figure  9.  Representative  Grading  Form 


simulator  TMS  terminal  and  integrated  with  the 
existing  data  in  the  trainee  history  file.  The 
training  selection  process  Is  initiated  and  the 
grading  form  for  the  next  simulator  familiari¬ 
zation  training  session  is  printed  out.  Its 
contents  and  corresponding  preparation  require¬ 
ments  are  briefly  reviewed  with  the  trainee 
(who  retains  the  form  for  subsequent  reference). 

The  instructional  event  is  concluded  by  placing 
the  completed  grading  form  in  the  trainee's  folder. 

OR AWING  THE  CONCLUSIONS. 

Although  brief  and  limited  to  somewhat  con¬ 
fined  areas,  the  Information  presented  should 
provide  a  sufficient  basis  to  allow  the  main 
Issues  identified  at  the  onset  to  be  re-addressed. 

Have  the  two  key  design  considerations  of 
operability  and  acceptability  been  satisfied? 

Would  the  design  approach  promote  ease  of  use? 

Is  ft  one  to  which  Instructors  could  relate? 

There  Is  a  third  and  equally  Important  consi¬ 
deration  as  well:  Is  it  realistically  achievable? 

How  wel 1  did  the  design  approach  conform  to 
the  twelve  objectives  and  guidelines  stated  at 
the  onset?  Specifically: 

1.  Were  the  number  of  Individual  controls 
and  functions  minimized? 

2.  Where  required  instructor  control 
actions  minimized? 

3.  Old  the  design  minimize  requirements 
to  enter  or  modify  variable  data? 

4.  Were  non-instructlonal  tasks  auto¬ 
mated  to  the  greatest  extent  possible? 

5.  Were  display  switching  requirements 
minimized? 


6.  Was  maximum  instructor  flexibility 
and  control  of  the  problem  provided? 

7.  Did  the  system  provide  continuous 
feedback  of  trainee  performance? 

8.  Was  performance  data  recording 
instructor  selectable? 

9.  Did  graphic  and  performance  data 
correlate  with  aircraft  .monitoring 
techniques  and  procedures? 

10.  Were  applicable  advanced  instructional 
features  made  available? 

11.  Did  exercise  preparation  minimize 
the  use  of  the  simulator? 

12.  Would  It  be  possible  for  new  instructors 
to  operate  this  system  totally  without 
referencing  a  handbook? 

One  remaining  issue  needs  to  be  resolved  be¬ 
fore  any  valid  conclusion  can  be  drawn.  How 
were  these  guidelines  and  objectives  selected, 
and  do  they  accurately  identify  the  critical 
design  areas  to  be  addressed?  Since  they  have 
all  (and  for  some,  repeatedly)  appeared  in 
military  procurement  specifications  in  the 
recent  past,  it  is  fairly  safe  to  assume 
that  they  do. 

SOME  FINAL  THOUGHTS. 

Effective  and  efficient  simulator  instructional 
systems  can  be  designed  and  produced  today,  but 
a  few  significant  trends  must  be  altered.  First 
is  the  specification  that  calls  for  minimum  In¬ 
structor  task  loading,  but  then  demands  maximum 
system  capability  combined  with  complete  and 
continuously  available  Instructor  control  of 
all  system  variables  and  training  features. 
Minimum  task  loading  is  normally  stated  as 
a  design  goal  and  maximum  control  as  a  require¬ 
ment.  Since  the  two  are  essentially  incompat¬ 
ible  when  specified  in  this  manner,  the  require¬ 
ment  must  obviously  take  precedence,  often  re¬ 
sulting  in  systems  that  are  characterized  by 
unnecessary  and  burdensome  operations.  This 
situation  is  further  compounded  by  the  erron¬ 
eous  tendency  to  measure  capability  and  train¬ 
ing  effectiveness  by  the  amount  of  Instructor 
control  provided.  These  and  other  similar 
trends  serve  only  to  mask  system  capabilities 
and  discourage  effective  utilization. 

The  answer,  therefore,  will  not  necessarily  be 
found  In  more  systems  and  more  technology,  but 
more  likely  from  a  willingness  on  the  part  of 
both  users  and  designers  to  make  the  true  dis¬ 
tinction  between  want  and  need  when  Identifying 
the  trainee.  Instructor  and  the  system  require¬ 
ments.  This  Is  a  matter  of  analytical  procedure 
tempered  by  self-discipline  —  a  process  that 
must  be  totally  oriented  toward  the  specific 
learning  objectives  of  each  system. 

THE  WRITER'S  CONCLUSION. 

This  paper  has  explored  and  expanded  on  some 
very  basic  design  concepts  emphasizing  practical 
application  and  common  sense.  It  has  offered  no 
really  new  or  Innovative  technology,  but  simply 
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proposed  some  logical  and  straightforward 
approaches  In  helping  to  resolve  some  rather 
old,  frustrating,  and  costly  problems.  It  Is 
not  the  answer,  but  If  your  response  to  some 


or  even  just  a  few  of  the  areas  addressed  was 
positive,  then  perhaps  It  Is  a  step  In  the 
right  direction  and  merits  further  consider¬ 
ation. 


THE  INSTRUCTIONAL  SUBSYSTEM,  A  MAJOR  COMPONENT 
OF  THE  TRAINING  DEVICE 


Dr.  Thomas  J.  Hamnell 
Eclectech  Associates,  Incorporated 
North  Stoning ton  Professional  Center 
North  Stonlngton,  Connecticut  06359 


ABSTRACT: 


The  Instructional  subsystem  has  tradition¬ 
ally  been  a  part  of  every  simulator/ 
training  device.  However,  It  has  often 
been  a  misnomer  consisting  of  little  more 
than  an  operator's  station  for  the 
controlling  of  the  sophisticated 
simulator.  The  necessary  technology 
currently  exists,  both  In  the  form  of 
hardware/software  capabilities  and 
tralnlng/lnstructlonal  techniques,  for  the 
development  and  application  of  potentially 
high  cost  effective  Instructional  tools  to 
enhance  the  Instructional  process.  Thus,  a 
"bonafide"  Instructional  subsystem  should 
be  a  part  of  every  training  device.  This 
paper  addresses  the  methodology  for 
developing  the  Instructional  subsystem, 
suggests  several  capabilities  of  that 
subsystem,  and  presents  an  applied  example 
of  an  Instructional  subsystem  In  the  form 
of  SMARTTS. 


INTRODUCTION 

The  General  Accounting  Office,  In  their 
report  to  Congress  concerning  "How  to 
Improve  the  Effectiveness  of  U.S.  Forces 
Through  Improved  Weapon  System  Design" 
(1981),  focused  on  the  Importance  of  the 
operator  to  the  overall  effective 
functioning  of  the  weapon  system,  and  the 
current  situation  of  Insufficient  early 
planning  to  provide  an  adequate  operator 
interface.  They  estimated  that  human 
errors  account  for  at  least  50  percent  of 
the  failures  of  major  weapon  systems.  They 
further  subdivided  these  failures  Into 
operator  skill  level  and  proficiency 
limitations,  amongst  other  factors.  Their 
findings  attribute  these  problems  to  the 
Increasing  complexity  of  modern  day 
systems.  Two  Important  Issues  are  evident 
from  this  recent  Investigation.  The  most 
obvious  Is  the  need  for  Improved  training, 
as  one  means  of  Improving  operator 
proficiency  and  reducing  operator  error. 
This  points  to  the  need  for  more  effective 
training  systems,  of  which  the  Instruc¬ 
tional  subsystem  1$  one  contributing 
aspect,  albeit  a  major  aspect.  Second,  the 
complexity  of  training  systems  Is 
Increasing  In  parallel  with  that  of  the 


operational  systems,  requiring  better  human 
factors  design  of  the  training  system 
Itself  (l.e..  Instructor  and  trainee 
Interfaces).  Both  of  these  Issues  point  to 
the  need  for  effective  human  factors  design 
of  the  training  system  from  an  Instruc¬ 
tional  process  standpoint. 

This  problem  of  developing  a  more  effective 
training  system  Is  not  really  new,  since 
the  training  community  has  always  been 
concerned  with  effective  Instruction.  The 
predominant  emphasis  in  the  design  of 
training  systems  in  recent  years,  however, 
has  been  placed  on  the  engineering  aspects, 
such  as  concern  over  adequate  simulation 
fidelity  and  the  technology  to  achieve  that 
fidelity.  Although  cost-effective  fidelity 
should  be  a  major  issue,  proportionate 
emphasis  should  be  given  to  the  other 
training-related  aspects  of  the  training 
device  and  system.  In  an  investigation  of 
the  training  effectiveness  differences 
attributable  to  differinq  levels  of 
fidelity  of  major  simulation  characteris¬ 
tics  (e.g.,  120  degrees  versus  240  degrees 
visual  scene  horizontal  field  of  view,  for 
a  shiphandling/shipbridge  simulator), 
instructor  differences  were  found  to  have 
several  times  the  impact  on  the 
effectiveness  of  training  in  comparison 
with  any  of  the  simulation  characteristics 
investigated  (Hammell,  Gynther,  Grasso,  and 
Gaffney,  1981).  The  relative  importance  of 
the  instructor  was  not  unexpected  (e.g., 
Caro,  1973),  although  its  strength  was 
surprising  in  the  presence  of  simulation 
characteristics  selected  for  their 
potential  impact  on  training  effective¬ 
ness.  The  importance  of  this  finding  is 
that  the  instructor  typically  embodies  most 
of  those  non-simulation  characteristics  of 
the  training  device  and  training  system 
(e.g.,  exercise  design,  monitoring  of 
student  performance,  student  feedback). 
The  Instructor  is  that  "catch-all"  that 
magically  transforms  the  simulator  into  a 
training  device.  This,  of  course,  should 
not  be  the  case.  The  training  device 
should  be  more  than  just  a  simulator.  It 
should  have  capabilities  specifically 
designed  to  augment  the  training  process. 

The  training  device  is  rapidly  becoming  a 
most  Important  element  of  the  training 
system,  and  is  often  the  centerpiece  of 
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that  training  system.  This  paper  focuses 
on  the  simulator-based  training  system, 
wherein  the  simulator/training  device  is  a 
major  element  of  that  system.  Other 
elements  Include  the  training  objectives, 
the  training  syllabus  and  other  training 
media  (e.g.,  at-sea  training).  The 
training  device,  as  It  has  often  been 
traditionally  known.  Is  nothing  more  than  a 
simulator.  It  merely  seeks  to  Imitate 
aspects  of  the  real  world,  such  as 
providing  a  radar  display  showing  Informa¬ 
tion  similar  to  that  which  would  be  seen  on 
the  actual  ship.  The  simulator  Is,  of 
course,  limited  In  that  which  it  can 
reproduce  faithfully.  Those  aspects  that 
are  simulated  are  presumably  those  deemed 
necessary  to  the  conduct  of  an  effective 
training  process,  while  many  others  are 
simply  not  addressed  by  the  simulator. 

"The  training  device  is  more  than  a 
simulator"  (Hammell,  1981).  Whereas  the 
simulator  simply  has  a  simulation  system, 
the  training  device  has  both  simulation  and 
training  subsystems.  The  simulation 
subsystem  is  that  noted  above.  The 
training  subsystem,  on  the  other  hand, 
should  consist  of  capabilities  that  are 
designed  specifically  to  enhance  the 
training  process  via  aiding  the  instructor, 
providing  information  to  the  students,  and 
so  on.  The  instructional  subsystem  (i.e., 
training  subsystem)  has  traditionally  been 
a  part  of  every  simulator/training  device. 
However,  i*  has  often  been  a  misnomer 
consisting  of  Tittle  more  than  an  operator 
station  for  the  controlling  of  the 
sophisticated  simulator;  it  has  seldom 
really  provided  capabilities  to  support  the 
trainirg  process.  The  necessary  technology 
currently  exists,  both  in  the  form  of 
hardware/software  capabilities  and 
training/instructional  techniques,  for  the 
development  and  application  of  potentially 
high  cost  effective  instructional  features 
to  enhance  the  instructional  process. 
Thus,  a  "bonafide"  instructional  subsystem 
should  be  a  part  of  every  training  device. 

The  instructional  subsystem  should  be 
tailored  based  on  the  many  considerations 
surrounding  the  particular  training 
application  and  the  training  device. 
Training  assistance  capabilities  should  be 
provided  to  support  the  instructor,  the 
trainee(s),  and  training  system  management 
(see  Figure  1).  The  instructor  support 
capabilities  should  provide  tools  and 
information  for  (1)  development  of  training 
exercises  and  support  materials,  (2) 
monitoring  and  control  of  the  training 
process,  and  (3)  assisting  in  achievement 
of  an  effective  training  process  interface 
with  the  trainee(s).  The  trainee  support 
capabilities  are  often  coincident  with 
"(3)".  Management  support  capabilities 


should  deal  with  inter  and  intra  site 
coordination,  the  continual  improvement  in 
training  cost/effectiveness,  and  so  on. 
Although  management  support  capabilities 
are  a  part  of  the  instructional  subsystem 
since  they  directly  impact  the  effective¬ 
ness  of  training,  they  will  be  only 
cursorily  addressed  herein. 

The  availability  of  sophisticated 
computer-display  technology  and  its 
traditional  incorporation  in  virtually  all 
sophisticated  simulator-based  training 
systems  enables  a  wide  variety  of  training 
assistance  technology  to  be  readily 
integrated  into  the  simulator-based 
training  device.  The  issue,  then,  is 
twofold:  (1)  instructional  technology 
should  be  incorporated  as  a  major  part  of 
every  training  device,  and  (2)  the  extent 
to  which  the  instructional  technology 
should  be  incorporated  depends  upon  the 
particular  training  objectives,  the 
capabilities  feasibly  available  on  the 
training  device,  and  the  many  other  issues 
and  constraints  surrounding  the  particular 
training  situation. 


Third  Generation  Training  System 

The  issues  currently  faced  in  the 
development  and  integration  of  instruc¬ 
tional  support  capabilities  represent  the 
start  of  the  third  generation  training 
device.  The  first  generation  training 
device,  which  is  typified  by  early  efforts 
at  simulation  (e.g.,  Link  Trainer)  had  as 
the  primary  concern  simulation  fidelity. 
The  problem  at  that  time  could  be  viewed  as 
one  of  simulation  fidelity  at  almost  any 
cost.  It  was  simply  necessary  to  achieve  a 
sufficient  amount  of  fidelity  in  the  early 
simulators  so  as  to  achieve  what  was 
considered  as  meaningful  training.  The 
primary  requisite  methodologies  and 
research  information  at  that  time  concerned 
engineering-related  issues;  that  is, 
engineering  design  and  simulation 
techniques.  These  issues,  of  course,  still 
remain  and  will  continue  as  relevant  issues 
for  the  development  of  simulators. 

As  adequate  simulation  capabilities  were 
achieved  in  several  areas,  the  second 
generation  of  training  devices  emerged. 
This  generation  was  no  longer  concerned 
with  achieving  adequate  simulation  at  any 
cost,  since  the  engineering  capabilities 
were  often  available.  Rather,  the  second 
generation  training  device  had  as  its 
primary  issue  cost  effectiveness  --  the 
cost  of  design  with  regard  to  the 
effectiveness  of  resultant  training. 
Rather  than  maximizing  simulation  fidelity 
at  any  cost,  the  major  considerations 
focused  on  the  minimal  level  of  simulation 


128 


fidelity  necessary  to  achieve  requisite 
training  effectiveness  (i.e.,  identifier 
tion  of  the  essential  features  of  the 
simulator,  and  their  respective  levels  of 
fidelity,  required  to  meet  specific 
training  objectives  and  performance 
standards).  The  variety  of  methodologies 
developed  to  address  this  issue  focused  on 
a  systematic  approach  to  the  design  of  the 
training  system  and/or  the  training 

device.  Examples  Include  the  Training 
Situation  Analysis  (TSA)  and  the 
Instructional  Systems  Development  approach 
(ISO).  These  methodologies  typically 
identified  the  specific  behavioral  tasks 
(i.e.,  including  skills,  knowledge,  and 

training  objectives)  of  the  trainee,  and 
then  sought  to  identify  the  minimum 

necessary  simulator/training  device 
characteristics  for  their  achievement.  As 
was  the  case  with  the  first  generation 
training  device  issues,  the  second 

generation  issue  (I.e.,  cost  effectiveness) 
remains  today,  and  will  continue  to  remain 
pertinent. 

The  training  industry  is  currently 
embarking  on  the  third  generation  training 
device.  As  the  second  generation  was  a 
further  refinement  of  the  first,  so  the 
third  generation  training  device  and  issues 
represent  a  further  refinement  of  the 
second  and  first  generations.  Whereas  the 
second  generation's  emphasis  was  on 
Identifying  the  minimally  acceptable  level 
of  simulation  fidelity  and  thus  minimizing 
the  cost  while  maintaining  effectiveness, 
the  third  generation  will  focus  on 
providing  training  enhancement  features  to 
greatly  Improve  the  effectiveness  of 
training  at  relatively  small  additional 
cost.  This  further  evolution  of  the 
training  device  will  augment  the  earlier 
generations  In  substantially  improving  the 
cost  effectiveness  of  training,  by 
specifically  designing  In  Instructional 
support  capabilities. 

Recognition  of  the  Importance  of  the 

Instructional  support  capabilities  has  been 
a  fundamental  problem,  but  one  that  Is 
being  steadily  overcome  in  both  the 
training  and  operational  communities.  The 
major  technological  problem  is  twofold: 
(1)  the  development  of  appropriate 

methodologies  and  resulting  research 
Information  pertaining  to  the  design  of  the 
instructional  subsystem,  and  (2) 
utilization  of  these  methodologies  and 

Information  to  actually  design  the 
Instructional  subsystem.  The  third 
generation  training  device  does  not,  as 
yet,  have  appropriate  methodologies 
adequately  developed,  much  less  an 

empirical  Information  base  from  which  to 
draw,  to  guide  the  design  of  the  instruc¬ 
tional  support  subsystem.  This  is  a 


substantial  problem  currently  facing  the 
training  device  industry. 

Methodologies  for  design  of  the 
instructional  subsystem  have  not  been 
formalized.  Hence,  the  instructional 
subsystem  has  traditionally  not  been 
included  as  a  part  of  the  training  device. 
Instructional  subsystems,  of  course,  have 
been  developed  and  implemented  on  certain 
devices,  although  to  an  extremely  limited 
extent.  The  instructional  subsystem  should 
consist  of  (1)  instructor  support 
capabilities,  (2)  trainee  interface 
capabilities,  and  (3)  training  system 
management  capabilities.  The  remainder  of 
this  paper  will  identify  aspects  of  these 
capabilities,  and  issues  to  be  considered 
in  their  development.  Since  one  of  the 
most  effective  means  of  instructing  is  that 
of  presenting  examples  to  the  students, 
this  paper  presents  an  example  of  an 
instructional  subsystem  known  as  SMARTTS 
(Submarine  Advanced  Reactive  Tactical 
Training  System),  which  has  been  recently 
implemented  on  a  Submarine  Combat  System 
Trainer  (SCST). 


INSTRUCTIONAL  SUBSYSTEM  CHARACTERISTICS  AND 
ISSUES 

The  instructional  subsystem,  as  noted 
above,  should  address  the  instructor,  the 
trainee,  and  system  management  (Figure  1). 
Each  of  these  are  considered  as  Important 
elements  of  the  instructional  subsystem 
(Hammell  and  Crosby,  1980;  Hammell,  1981). 
This  paper,  focuses  particularly  on 
capabilities  to  support  the  instructor  and 
trainee. 

The  Submarine  Advanced  Reactive  Tactical 
Training  System  (SMARTTS)  (Hammell  and 
Crosby,  1980)  is  a  pioneering  effort  in  the 
development  of  instructional  support 
capabilities,  in  that  it  is  currently 
Integrating  a  sophisticated  Instructional 
support  subsystem  with  an  existing 
submarine  combat  systems  trainer.  This 
project  developed  a  substantial  body  of 
information  regarding  the  design  and 
application  of  the  instructional  subsystem, 
and  may  be  used  as  a  departure  point  for 
directly  addressing  many  relevant  Issues. 
The  SMARTTS  project  has,  for  example,  (1) 
developed  and  applied  some  of  the 
potentially  useful  techniques  for  the 
generation  of  Information  to  design 
instructional  support  characteristics,  (2) 
has  designed  a  variety  of  Instructional 
support  characteristics  which  are  currently 
being  Implemented,  and  (3)  Is  planning  to 
embark  on  a  test  and  evaluation  of  the 
Instructional  support  subsystem  and  its 
particular  set  of  characteristics.  The 
problems  identified,  the  experiences,  and 
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the  techniques  developed  during  the  SMARTTS 
project  provide  considerable  insight  into 
many  specific  issues  relevant  to  the 
instructional  subsystem. 

The  Instructional  subsystem  is  addressed 
below  in  terms  of  (1)  design  methodology, 
which  discusses  the  overall  approach  to  the 
design  of  the  instructional  subsystem;  and 
(2)  training  assistance  technology,  the 
specific  capabilities  of  the  instructional 
support  subsystem.  Several  issues  are 
addressed  under  each  of  these,  with 
examples  drawn  from  SMARTTS  as  appro¬ 
priate.  Also,  concepts  for  future 
development  are  introduced. 


Design  Methodology 

Highly  structured  methodologies  are 
available  for  the  selection  and  design  of 
training  media  (e.g..  Training  Situation 
Analysis  (TSA),  Chenzoff,  1965; 
Instructional  System  Development  ( ISO) , 
NAVEDTRA  106A).  These  methods,  which  have 
been  demonstrated  as  effective,  are 
primarily  intended  to  address  the 
student/trainee  interface  with  the  training 
media.  For  example,  a  major  use  of  the  ISO 
approach  is  to  determine  simulator/training 
device  fidelity  characteristics  (e.g., 
level  of  visual  scene  fidelity).  These 
systematic  approaches  relate  the  training 
device/training  system  characteristics  to 
specific  trainee  operational  tasks.  This 
of  course  is  necessary.  However,  such  an 
approach  does  not  directly  address  those 
training  device/training  systems  capabil¬ 
ities  that  can  directly  augment  the 
training  process  but  are  not  directly 
linked  to  the  operational  tasks  (e.g., 
external  feedback).  These  other 
training-related  characteristics  do  not 
derive  from  the  operational  tasks,  but 
rather  derive  from  the  base  of  generic 
training  research  and  methodologies.  There 
is  obviously  a  need  to  design  the  training 
system  with  regard  to  both  operational 
tasks  and  generic  training  methodologies. 
In  practice,  both  are  addressed,  although 
the  latter  is  not  done  so  formally. 


INSTRUCTOR  INTERFACE  ANALYSIS.  The 
available  methods  have  not  adequatel y 

addressed  the  Instructor  Interface  (i.e., 
including  various  forms  of  Instructor  aids) 
to  the  training  device/training  system. 
Relatively  little  emphasis  has  typically 

been  placed  on  the  design  characteristics 

of  the  training  device/training  system  to 
directly  assist  the  Instructor  in 

conducting  the  training  process.  Needed 
Is  a  structured  approach  for  the  design  of 
the  Instructional  subsystem.  The  analysis 
should  be  quite  similar  to  that  of  ISO, 


although  not  focusing  on  operational  tasks, 
but  rather  focusing  on  instructor  tasks  and 
training  methodological  principles.  This 
type  of  analysis  has  been  employed  in  the 
past,  although  to  a  very  limited  extent 
(e.g.,  the  SMARTTS  example  presented  later; 
also,  Charles,  1978).  Several  elements  of 
an  ISO  process  for  the  instructional 

subsystem  might  include  the  following 

elements: 

•  Instructor  task  analysis  —  a 

detailed  analysis  of  the  functions 
and  tasks  to  be  performed  by  the 
instructor  on  all  aspects  associated 
with  his  training  role.  This 

analyses  should  form  the  basis  for 
the  design  of  training  system/- 
training  device  characteristics  to 
support  the  instructor  in  conducting 
an  effective  training  process. 

•  Analysis  of  instructor  loading  --  an 
estimate  should  be  made  of  the  load 
placed  on  the  instructor  in  the 
performance  of  his  functions  and 
tasks.  This  load  should  address  for 
each  task  the  difficulty,  time  to 
perform,  frequency,  available 
resources,  accuracy  of  performance 
required,  and  so  on.  The  intention 
is  to  develop  an  accurate  profile  of 
the  loading  placed  on  the  instructor 
across  his  functions  and  tasks,  and 
his  expected  performance  level  on 
the  basis  of  the  training  system 
design. 

•  Identification  of  critical 

instructor  functions  and  tasks  — 
based  on  the  above  two  elements, 
those  instructor  functions  and  tasks 
that  would  benefit  substantially 
from  other  assistance  should  be 
identified.  For  example,  data 
recording  is  a  task  that  a  good 
instructor  would  perform  frequently, 
so  as  to  have  good  postexercise 
feedback  information.  Data 

recording  can  be  extremely  cumber¬ 
some  and  time  consuming;  further¬ 
more,  it  is  a  task  that  can  often  be 
readily  performed  by  the  computer/- 
training  device,  thus  off  loading 
the  instructor  to  devote  more  of  his 
time  to  those  tasks  that  only  he  can 
adequately  perform  (e.g.,  monitoring 
student  communication). 

•  Determine  instructor  support 

capabilities  —  specific  instructor 
support  capabilities  should  be 
determined  via  a  cost/effective 

trade-off  analysis  based  on  the 
above  elements,  alternative  design 
approaches  for  providing  support 
capabilities,  and  their  expected 


effectiveness  In  assisting  the 
instructor  and  enhancing  the 
effectiveness  of  the  training 
process.  A  media  selection  approach 
to  support  the  Instructor  functions 
and  tasks  should  be  performed, 
similar  to  that  accomplished  under 
ISO  for  the  trainee. 

This  type  of  front-end  analysis  is  devoted 
to  the  instructor,  and  should  result  in  the 
generation  of  cost/effective  training 
device/training  system  characteristics. 
Although  aimed  at  specifically  supporting 
the  instructor,  their  usefulness  is  based 
on  enhancing  the  training  process  with 
regard  to  cost/effective  considerations. 
Many  of  these  characteristics  will  also 
directly  impact  the  trainee  interface  as 
well,  since  many  of  the  instructor  tasks 
deal  with  the  trainee  interface. 


TRAINING  METH000L0GY  AIDS  ANALYSIS.  A 
similar  analysis  should  be  conducted  to 
determine  support  capabilities  with  regard 
to  generic  training  methodologies  (e.g., 
importance  of  specific  external  feedback). 
This  type  of  analysis,  which  would  impact 
instructor  and  trainee  interfaces,  would 
have  a  similar  series  of  elements,  as 
follows: 

•  Training  methodology  characteristics 
—  identification  of  those  specific 
training  process  characteristics 
that  are  relevant  to  the  particular 
training  situations  supported  by  the 
training  device/training  system 
(e.g.,  immediate  graphical  feedback 
of  tactical  parameters).  This  would 
be  similar  to  a  task  analysis,  but 
conducted  to  identify  the  specific 
training  methodology  characteris¬ 
tics.  The  intention  is  to  Identify 
all  those  characteristics  of  the 
generic  training  methodologies  that 
may  be  important  for  this  particular 
training  process. 

•  Prioritization  of  training 

methodology  characteristics  —  each 
of  the  training  methodology 
characteristics  should  be  evaluated 
with  regard  to  Its  potential 
effectiveness  during  the  training 
process,  for  the  particular  training 
device/training  system  under 

consideration.  The  result  would  be 
a  prioritization  of  those  character¬ 
istics  on  the  basis  of  their  likely 
training  effectiveness  Impact. 

•  System/device  capability  analysis  — 
the  capability  of  the  training 
device/training  system  to  support 
each  of  the  training  methodology 


characteristics  should  be  deter¬ 
mined.  This  should  be  done  with 
regard  to  the  specific  aspects  of 
the  training  situation  and 
methodology  characteristics  desired 
to  achieve  an  effective  training 
process.  For  example,  if  a  complex 
tactical  problem  is  being  trained, 
delayed  feedback  regarding  detailed 
tactical  and  performance  parameters 
may  be  desired  to  enable  detailed 
analytical  Investigation  of 
parameter  interrelationships;  the 
ability  of  the  training  system/- 
training  device  to  provide  the 
detailed  feedback  information  in  an 
appropriate  form  (e.g.,  on  a  large 
screen  graphical  display)  should  be 
evaluated. 

•  Identify  instructional  support 
capabilities  —  based  on  the  above 
elements  and  a  cost/effectiveness 
trade-off  analysis,  specific  support 
capabilities  would  be  identified. 
The  results  could  identify  deficient 
capabilities  on  the  existing 
training  device/training  system,  or 
could  identify/select  capabilities 
to  be  developed  in  a  new  training 
device. 

The  above  elements  represent  an  outline  of 
steps,  similar  to  that  of  the  ISD  process, 
for  the  design  of  the  instructional 
subsystem  characteristics.  Obviously,  all 
of  the  steps  have  not  been  detailed,  nor 
have  their  specific  procedures  been 
adequately  developed.  Rather,  they  are 
intended  to  indicate  that  a  structured 
process  similar  to  that  of  ISO  should  be 
accomplished  for  each  of  the  major  aspects 
of  the  Instructional  support  subsystem. 
Furthermore,  the  instructional  support 
subsystem  includes  several  major  parts  of 
the  training  device/training  system,  each 
of  which  requires  a  distinct  analysis.  It 
Is  Important  to  note  that  the  above 
analyses  should  be  adequately  developed, 
but  done  so  quickly  with  a  minimum  of 
frills.  The  analysis  should  be  tailored  to 
the  available  Information,  the  sophistica¬ 
tion  of  the  training  device/training 
system,  and  the  available  resources.  Many 
of  the  characteristics  could  be  evaluated 
from  the  standpoint  of  a  shopping  list  of 
potentially  useful  charcteristics;  and  the 
final  set  selected  from  the  list. 

This  approach  Is  appropriate  for  the  design 
of  a  new  training  device/training  system, 
as  well  as  that  of  Improving  an  existing 
one.  SMARTTS  is  an  example  of  the  latter 
case.  The  methodologies  are  likely  to  be 
slightly  different  for  each  case,  but 
basically  the  same. 
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SMARTTS  DESIGN.  The  SMARTTS  Instructional 
support  subsystem  design  was  the  result  of 
several  analyses  similar  to  the  above,  over 
a  period  of  years.  The  initial  concepts  of 
SMARTTS  stems  from  an  investigation  of 
submarine  tactics  training  (Hanmell,  Sroka, 
and  Allen,  1971;  Hanmell,  Gasteyer,  and 
Pesch,  1973).  Of  the  variety  of 
recommendations  that  were  developed 
regarding  basic  and  advanced  submarine 
officer  training  systems,  many  addressed 
the  need  for  appropriate  instructional 
support  capabilities.  Figure  2,  taken  from 
the  latter  report,  lays  out  the  fundamental 
instructional  support  capabilities  that 
were  later  to  be  developed  under  SMARTTS. 
The  concepts  developed  in  these  earlier 
investigations  were  later  fully  developed 
in  specific  detail  on  the  basis  of  a  front 
end  analysis  conducted  under  the  SMARTTS 
project.  This  later  analysis  included 
several  of  the  above-recommended  steps, 
focusing  on  both  the  instructor  tasks  and 
generic  training  methodologies.  This 
analysis  also  selected  which  of  the 
capabilities  recommended  in  the  earlier 
studies  were  to  be  developed  in  the  initial 
preprototype  SMARTTS,  and  which  will  be 
developed  in  subsequent  developmental 
versions  of  SMARTTS. 

An  overview  of  the  instructor  function  and 
task  analysis  conducted  for  SMARTTS  is 
presented  below.. 


Instructor  Function  and  Task  Analysis.  An 
overview  of  the  analysis  of  instructor 
functions  and  tasks  is  presented  in  Figures 
3,  4,  5,  and  6.  This  analysis  identified 
the  major  functions  conducted  by  the 
submarine  tactics  instructor,  the  specific 
tasks  he  performs  with  regard  to  each 
function,  and  estimated  the  time  loading 
and  difficulty  associated  with  performance 
of  those  tasks.  The  major  instructor 
functions  were  determined  as  follows: 

•  exercise  development 

•  monitor  and  control  of  the  training 
process 

•  briefing  (pre,  freeze,  post) 

•  training  system  management 

Monitor  and  control  of  the  training  process 
is  the  instructor  function  given  the 
greatest  recognition  by  training  device 
builders  (Figure  4).  When  a  training 
device  is  employed,  the  instructor  must 
set-up  the  exercise,  control  the  device 
during  the  exercise  (e.g.,  maneuvering 
targets),  monitor  the  trainees'  activities, 
and  provide  some  amount  of  training  during 
the  exercise  (e.g.,  guidance  and  feedback 
to  the  trainees).  Capabilities,  in  one 
form  or  another,  are  provided  on  most 
training  devices  to  enable  set  up  and 


control  of  the  exercise.  Some  capabilities 
are  often  provided  to  permit  monitoring  of 
trainee  activities,  and  often  some 
recording.  Occasionally,  capabilities  are 
available  for  providing  some  training 
assistance,  such  as  feedback.  However,  all 
too  often  the  monitoring  and  control 
capabilities  installed  on  a  training  device 
are  designed  from  an  engineering-control 
standpoint,  rather  than  from  an  instruc¬ 
tional  process  control  viewpoint.  Specific 
instructor  tasks,  A-L,  are  included  in 
Figure  4,  along  with  generic  training 
issues  and  considerations  under  each  task. 
Also,  the  multiplicity  of  branching  paths 
between  tasks  is  also  indicated  to  some 
extent  by  the  (A)  and  (§)  symbols.  The 
information  contained  in  Figure  4  is  a 
summary  of  the  type  of  analysis  information 
upon  which  the  SMARTTS  characteristics  were 
based.  For  example,  the  instructor  has  a 
task  to  monitor  the  scenario  (i.e.,  Task 
C).  A  consideration  under  that  task, 
particularly  in  a  complex  system,  is 
providing  cues  to  the  instructor  regarding 
current  and  upcoming  scenario  events. 
Capabilities  were  developed  in  SMARTTS  to 
provide  cues  and  alerts  to  the  instructor 
regarding  various  tactical  actions  on  the 
part  of  the  target,  as  well  as  when  various 
performance  indicators  (e.g.,  probability 
of  ownship  counterdetection)  went  beyond 
preset  standards.  These  capabilities, 
which  are  easily  provided  by  the  computer- 
controlled  training  device,  reduce  the 
instructor’s  scenario  monitoring  load, 
enabling  him  to  devote  a  greater  proportion 
of  time  to  monitoring  the  trainees  (i.e.. 
Task  H).  Other  examples  are  given  later 
under  Training  Assistance  Technology. 

Briefing  of  the  trainees  (Figure  5),  in  its 
various  forms,  is  probably  the  most 
important  function  performed  by  the 
instructor.  In  this  function  the 
instructor  has  a  direct  interface  with  the 
trainees,  and  provides  them  with  specific 
information  to  reinforce  and/or  modify 
their  behavior.  In  a  simulator-based 
training  system  briefings  may  be  given 
prior  to,  during  a  pause  in,  and  following 
completion  of  the  session  on  the  simulator 
(i.e.,  pre,  freeze,  and  post  briefings 
respectively).  Although  essential  to  the 
effectiveness  of  the  training  process,  the 
emphasis  placed  on  briefings  varies 
considerably  across  training  establish¬ 
ments.  Furthermore,  the  capabilities 
provided  as  part  of  the  training  device/- 
training  system  to  assist  in  conducting 
effective  briefings  are  often  quite 
limited.  Examples  of  capabilities  to 
support  briefing  tasks  will  be  provided 
later  below  under  Training  Assistance 
Technology. 
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TRAINING  SYSTEM 


F Inure  1.  Focus  of  Instructional  Subsystem 


Figure  2.  Instructional  Support  Capability  Requirements 
for  the  21 A  Series  Submarine  Combat  System  Trainers 
(from  Hanwell,  Gasteyer  and  Pesch,  1973). 
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Flour*  3.  Exercise  Development 
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Floure  4.  Monitor  and  Control  Trelnlno 


Figure  5.  Briefing  (Pre,  Freeze.  Pott) 


Exercise  development  (Figure  3),  which 
proceeds  the  conduct  of  training  sessions, 
consumes  a  substantial  portion  of  an 
Instructor’s  time.  This  Is  a  complex 
function  wherein  the  Instructor  actually 
develops  the  training  program  and  Its 
supporting  materials.  It  is  complex  In 
that  It  often  requires  the  creative  design 
of  the  course  and  Its  training  exercises, 
with  the  Instructor  drawing  upon  the 
state-of-the-art  In  training  methodology  to 
achieve  an  effective  training  process. 
Whereas  some  capabilities,  although  usually 
quite  limited,  are  often  provided  on  the 
training  device/training  system  to  support 
exercise  monitoring,  control  and  briefing, 
usually  little  If  any  capabilities  are 
provided  to  support  exercise  development. 
It  Is  the  exercise  development  function, 
interestingly,  that  determines  how  the 
training  device  will  be  employed  within  the 
training  system.  Examples  of  system 
capabilities  to  support  exercise  develop¬ 
ment  are  presented  later  under  Training 
Assistance  Technology. 

Training  system  management,  similar  to 
exercise  development,  is  typically  an 
overlooked  function  of  the  instructor,  and 
one  with  which  he  spends  substantial  time. 
A  wide  variety  of  tasks,  issues,  and 
considerations  may  be  associated  with 
training  system  management.  Furthermore, 
each  of  these  may  depend  on  the  particular 
training  establishment  under  which  the 
training  device/training  system  operates. 
Nevertheless,  several  training  system 
management  tasks  are  common  across  training 
establishments,  as  indicated  in  Figure  6. 
Perhaps  the  most  important  common  set  of 
tasks  involves  monitoring  the  effectiveness 
of  training  over  time.  For  example,  it  is 
highly  desirable  to  identify  those  training 
objectives  (i.e.,  with  associated 
exercises)  that  the  typical  trainees  can 
readily  perform  upon  entering  the  training 
program;  and  likewise  to  identify  those 
training  objectives  (and  exercises)  that 
the  trainees  typically  have  substantial 
difficulty  with,  even  near  the  end  of  the 
training  program.  Ideally,  reduced 
emphasis  would  be  placed  on  the  former 
while  increased  emphasis  would  be  placed  on 
the  latter  training  objectives  in  subse¬ 
quent  training  programs.  For  another 
example,  alternative  training  methods  and 
training  materials  should  be  periodically 
evaluated  to  continually  upgrade  their 
quality  and  the  effectiveness  of  the 
training  process.  These  are  major 
management  tasks  directly  impacting  the 
cost/effectiveness  of  training,  and 
subsequently  the  operational  readiness  and 
effectiveness  of  weapon  systems. 

Each  time  a  training  exercise  is  run  on  the 
simulator  valuable  training  performance 


data  Is  generated.  These  data  could  be 
collected  over  time  and  used  as  the  basis 
for  the  above  evaluations,  and  other 
training  system  development  activities. 
These  data  would  be  extremely  useful  to 
most  levels  of  management  for  evaluation 
and  planning  purposes.  Relatively  little 
of  these  data  are  recorded  and  used  today 
with  regard  to  most  training  device/- 
tralnlng  systems.  This  is  one  example  of  a 
set  of  training  device/training  system 
capabilities  that  could  substantially 
augment  training  system  management. 

The  Instructor  function  and  task  analysis 
conducted  on  SMARTTS  was  integrated  with 
the  generic  training  methodology  analysis 
as  indicated  in  Figures  3  through  6.  This 
analysis  resulted  in  the  identification  of 
issues  and  constraints  deemed  important  to 
the  conduct  of  an  effective  training 
process  for  submarine  officer  tactics 
training.  These  form  the  basis  for  the 
subsequent  design  of  the  SMARTTS  preproto¬ 
type  system  (i.e.,  the  instructional 
support  subsystem  capabilities  which  make 
up  SMARTTS).  A  similar  process  should  be 
conducted  as  part  of  the  design  for  each 
training  device/training  system. 

The  next  step  of  the  design  process  is  the 
trade-off  analysis  to  identify  and  design 
the  specific  capabilities  to  be  included  in 
the  particular  instructional  support 
subsystem  for  the  particular  training 
device/training  system.  This  trade-off 
analysis  should  be  conducted  similar  to 
that  of  other  cost/effective  analyses  done 
under  the  ISD  process.  The  results  of  the 
analysis  conducted  for  SMARTTS,  although 
not  the  analysis  itself,  will  be  summarized 
later  in  this  paper  under  Training 
Assistance  Technology. 


Other  Design  Issues.  The  above  recommended 
analysis  has  as  its  objective  the 
development  of  the  characteristics  of  the 
Instructional  subsystem  from  the  stand¬ 
points  of  the  instructor  functions  and 
application/enhancement  of  effective 
training  methodologies.  Several  other 
issues  should  also  be  considered  during  the 
design  of  this  subsystem.  Of  primary 
importance  is  the  flexibility  of  the 
instructional  subsystem  with  regard  to 
future  upgrading.  That  is,  the  subsystem 
should  be  designed  with  evolution  in  mind. 
As  the  training  device/training  system  is 
used,  and  as  information  is  generated 
regarding  the  effectiveness  of  its  various 
charcteristics,  the  instructional  subsystem 
should  be  continually  improved.  More 
effective  training  methodologies  should  be 
incorporated,  new  performance  indicators 
and  feedback  displays  developed,  and  so 
on.  As  the  instructors  continually 


r 


r 


rJ 

i 


136 


Increase  their  experience  In  training  and 
with  the  devlce/system,  as  the  operational 
systems  continually  change,  and  as  the 
trainees  continually  Improve  their 
performance  the  training  device/training 
system  should  likewise  change  to  refocus 
training  emphasis  and  to  continually 
Improve  the  effectiveness  of  training.  The 
flexibility  to  permit  evolution  In  this 
regard  must  be  designed-in  at  the  onset, 
with  a  conscious  effort  made  to  continually 
upgrade  training.  The  engineering 
characteristics  of  the  system  should  be 
such  that  the  major  training-related 
characteristics  can  be  readily  modified, 
and  new  characteristics  added.  This 
Impacts  the  design  of  the  hardware 
components  and  the  software  architecture. 
For  example,  new  performance  indicators  and 
feedback  displays  will  always  be  required. 
Hence,  the  software  architecture  should  be 
designed  to  permit  easily  adding  new 
performance  Indicator  algorithms,  which  may 
require  access  to  a  wide  variety  of 
scenario  parameters;  generic  display 
formats  should  be  set  up  which  enable  the 
construction  of  new  displays  drawing  upon 
the  variety  of  parameters  generated  and 
recorded,  and  a  variety  of  graphical 
formats.  Although  details  of  system  design 
flexibility  are  not  discussed  In  this 
paper,  they  have  been  incorporated  Into  the 
SMARTTS  system.  It  Is  anticipated  that  as 
SMARTTS  Is  used  demand  for  new  capabilities 
will  continually  exist.  The  SMARTTS 
software  architecture  is  designed  to 
readily  accommodate  such  modifications  and 
additions  as  relatively  minor  software 
changes.  Some  of  the  capabilities 
identified  under  the  SMARTTS  front-end 
analysis  that  would  further  enhance  this 
evolution/modification  process  have  not 
been  Included  In  the  preprototype,  but  are 
obviously  considered  for  the  further 
developmental  units. 

It  Is  apparent,  as  Indicated  above,  that 
the  training  process  conducted  using  a 
particular  training  device/training  system 
will  change  over  time.  Furthermore,  in  a 
similar  vein,  at  any  given  point  In  time 
the  complex  simulator-based  training  system 
Is  likely  to  be  used  for  a  wide  range  of 
training  levels  and  training  needs.  Hence, 
system  flexibility  should  also  address  the 
ready  tailoring  of  the  Instructional 
support  capabilities  to  a  particular 
training  situation.  That  is,  different 
types  of  support  capability  characteristics 
are  required  for  the  different  levels  of 
training  likely  to  be  encountered.  Many  of 
these  can  be  determined  In  advanced,  while 
others  must  be  tailored  close  to  the  time 
of  Initiating  the  training  exercise.  The 
training  device/training  system,  and  Its 
Instructional  support  subsystem  should  be 
flexibly  designed  to  permit  on  the  spot 


tailoring  of  the  training  exercise, 
performance  Indicators  used,  feedback 
displays  and  their  formats,  and  so  on.  For 
example,  it  may  be  necessary  to  generate/- 
modify  the  next  training  exercise 
specifically  on  the  basis  of  performance  In 
the  previous  training  exercise.  Also,  the 
performance  standard  on  which  a  cue  Is 
presented  to  the  Instructor  may  likewise 
have  to  be  modified  shortly  prior  to 
running  the  exercise.  The  system 
architecture  should  be  flexible  to  enable 
this  type  of  tailoring  with  relatively 
little  effort. 

As  noted  In  the  Introductory  paragraph  to 
this  paper,  human  factoring  of  the 
instructor  and  trainee  Interfaces  is  as 
important  in  the  training  device/training 
system  as  it  is  in  the  operational  weapons 
system.  Too  often  this  aspect  of  design  Is 
overlooked.  Many  of  the  instructional 
support  capabilities  provided  to  the 
instructor  could  simply  be  included  under 
the  category  of  "good  human  factors 
design",  since  their  main  purpose  is  to 
facilitate  the  instructor/system  interface 
and  improve  the  effectiveness  of  his 
performance.  This  includes,  for  example, 
the  specific  human  computer/training  device 
interface  for  exercise  set  up,  control  and 
monitoring.  An  effort  should  be  made  to 
minimize  the  amount  of  training  necessary 
for  the  instructor  to  be  able  to  operate 
the  training  device.  On  SMARTTS,  for 
example,  the  traditional  alphanumeric/- 
functlon-key  input  device  was  replaced  with 
a  touch-sensitive  plasma  display  device. 
The  plasma  input  device  has  many  advantages 
beyond  those  which  have  been  incorporated 
into  the  preprototype  SMARTTS;  neverthe¬ 
less,  those  plasma-related  input  features 
incorporated  into  SMARTTS  should  provide  a 
substantial  increase  in  the  effectiveness 
with  which  the  instructor  can  communicate 
with  the  training  device.  The  instructor 
does  not  have  to  learn,  for  example,  a 
large  number  of  input  codes  for  effecting 
control  of  the  training  device;  he  does  not 
have  to  remember  the  options  available  to 
him,  and  look  up  the  appropriate  codes,  or 
coordinate  the  options  from  the  CRT  to  the 
alphanumeric  input  keyboard;  he  does  not 
have  to  input  a  sequence  of  alphanumeric 
commands  on  the  keyboard.  Rather,  the 
complete  set  of  Input  commands  are 
logically  layed  out  in  a  tree  structure, 
and  only  those  commands  available  to  him  at 
any  point  in  time  are  displayed  on  the 
plasma  entry  device.  These  available 
commands  are  displayed  in  English.  To 
enter  a  command,  he  simply  points  with  his 
finger  to  the  appropriate  command.  This 
type  of  interface  is  user  friendly,  and  to 
a  large  extent  self- teaching  through  use  of 
the  device  itself.  Careful  consideration 
should  be  given  to  the  many  other  aspects 
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of  the  Instructor  and  trainee  Interfaces 
with  the  training  device  (e.g.,  location  of 
Instructor  Information,  location  of  trainee 
feedback  displays). 


Training  Assistance  Technology 

The  remainder  of  this  paper  Identifies 
aspects  of  training  assistance  technology 
that  are  important  elements  of  the 
instructional  support  subsystem.  These  are 
presented  with  regard  to  each  of  the  four 
major  instructor  functions  identified 
above.  Specific  examples  of  characteris¬ 
tics  are  given  pertaining  to  the  SMARTTS 
system,  where  applicable. 


EXERCISE  DEVELOPMENT  CAPABILITIES.  (Figure 
7J1  txercfse  development  may  include  the 
development  of  the  training  objectives,  the 
course  syllabus,  and  so  on.  Although  these 
capabilities  can  be  supported  by  parts  of 
the  Instructional  subsystem,  under  this 
function  and  under  the  training  system 
management  function,  they  are  not  directly 
addressed  herein.  Rather,  those 
capabilities  more  directly  related  to  the 
training  device  activities  itself  (i.e., 
conduct  of  scenario  exercises)  are 
specifically  addressed  herein,  and 
summarized  in  Figure  3. 


Instructional  Support  Material .  A  variety 
of  material  should  Be  available  to  the 
Instructor  as  his  primary  resource  for 
developing  the  training  device  exercise 
scenario.  Most  of  this  material  would 
likely  be  in  the  form  of  handbooks, 
although  some  could  be  available  in  a 
computer  data  base.  This  information 
should  Include  (1)  a  complete  set  of 
tactical  and  behavioral  training 
objectives,  cross-referenced  to  exercises 
and  operational  situations;  (2)  a  complete 
set  of  tactical  reference  information,  such 
as  Naval  Weapons  Publication  (NWP)  series 
documents;  (3)  a  set  of  guidelines 
regarding  training  methodologies,  training 
device  operations,  and  training  exercise 
development;  and  (4)  trainee  input 
characteristics  information.  Including 
previous  trainee/team  performance 
Information,  and  training/tactical  needs. 
The  Information  related  to  trainee  Input 
characteristics  could  be  available  on  a 
large  Information  base  assembled  from 
previous  training  exercises  of  that 
particular  Individual  or  team,  together 
with  comparative  data  on  the  population  of 
similar  trainees,  and  so  on.  The  necessary 
data  would  be  collected  under  the  training 
system  management  function  of  the 
instructor. 


The'  SMARTTS  system,  to  support  this 
exercise  development  function,  has 
developed  documents  to  aid  the  instructor 
in  the  design  of  exercises.  SMARTTS  has 
developed  not  only  the  standard  system 
operating  handbook,  but  also  an 
instructor's  handbook  which  addresses  how 
to  design  exercises,  alternative  training 
methods  that  can  be  employed,  how  to 
conduct  a  training  exercise,  and  so  onl 
Additionally,  a  comprehensive  set  of 
training  objectives  have  been  developed 
along  with  a  set  of  training  exercises 
cross-referenced  to  these  objectives.  A 
training  structure  has  been  developed 
across  multiple  levels  of  trainee 
proficiency,  with  diagnostic  information 
supplied  to  evaluate  the  trainee/team  input 
characteristics;  remedial  training 
activities  have  also  been  included  to  help 
assure  that  all  trainees  meet  the  mininum 
entry  level  standards.  Furthermore, 
training  materials  for  the  initial  tactical 
course,  including  exercises,  have  been 
developed  utilizing  the  SMARTTS  capabil¬ 
ities.  This  approach  not  only  has  provided 
the  structure  and  guidelines  with  which 
instructors  can  develop  subsequent  training 
courses  and  exercises,  but  has  also 
provided  an  example  in  the  form  of  the 
initial  such  course. 


Exercise  Development  Simulation.  A 
flexible  high  speed  simulation  capability 
is  necessary  to  enable  the  instructor  to 
review  existing  exercises  in  the  library, 
modify  an  exercise,  or  create  a  new 
exercise.  This  fast  time  simulation 
capability  is  necessary  in  support  of  tasks 
"B"  through  "I"  under  the  exercise 
development  function  (Figure  3).  Review, 
modification,  or  development  of  an  exercise 
can  be  an  extremely  cumbersome  and  time 
consuming  process.  This  is  particularly 
the  case  where  calculation  of  the 
developing  time  line  scenario  interaction 
is  necessary  to  assure  appropriate  scenario 
events  in  support  of  the  exercise 
objectives.  Furthermore,  it  is  well 
recognized  that  scenarios  often  turn  out 
substantially  different  when  run  on  the 
device  than  as  they  appeared  on  the  desk 
top  plot  sheet  of  the  developer.  The  fast 
time  simulation  should  provide  capability 
for  the  instructor  to: 

(1)  call  up  an  existing  exercise  and  run 
it  in  fast  time  to  any  desired 
point,  retrace  steps,  and  so  on; 

(2)  investigate  alternative  actions  from 
any  desired  time  to  any  other 
desired  time; 

(3)  generate  and  display  relevant 
performance  indicators  and  situation 
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parameters  (e.g.,  target  range)  at 
the  various  times  of  the  actual 
scenario  and  the  alternatives  under 
Investigation; 

(4)  modify  any  of  the  scenario 
parameters,  enabling  comparison  of 
the  tactical  parameters  and 
performance  Indicators  at  any 
subsequent  point  In  time; 

(5)  enter  new  performance  Indicator 
algorithms  to  be  recorded  and 
displayed  during  or  subsequent  to 
the  exercise; 

(6)  Insert  Instructor  cues  to  be 
automatically  keyed  at  times  during 
the  exercise  based  on  various 
aspects  of  the  scenario  parameters 
and/or  time; 

(7)  configure  new  feedback  display 
formats  for  trainee  briefings  based 
on  the  training  objectives.  Issues 
to  be  focused  on,  or  parameters 
generated  during  the  scenario,  and 
so  on; 

(8)  develop  new  subjective  performance 
monitoring  Input  categories  for 
observation  entry  by  the  Instructor 
during  the  actual  exercise; 

(9)  set  up  all  necessary  scenario 
parameters  for  running  the  scenario 
on  the  training  device; 

(10)  permanently  record  the  exercise  in 
the  exercise  library  for  later 
review  via  this  fast  time  simulation 
capability,  or  for  actual  running 
during  a  training  exercise  on  the 
training  device. 

Additional  capabilities  could  also  be 
listed;  but  these  present  a  good  overview 
of  the  type  of  capabilities  that  should  be 
available  to  the  instructor  In  an  off-line 
mode  of  the  training  device  to  enable  him 
to  review  and  configure  scenarios.  A 
library  of  all  available  training  exercise 
scenarios  should  obviously  be  accessible  to 
the  Instructor  for  this  development 
process.  This  fast  -  time  simulation 
capability  should  be  available  at  an 
off-line  location  to  the  training  device, 
permitting  access  and  fast  time  simulation 
In  parallel  with  the  actual  running  of  the 
training  device.  The  terminal  should  have 
appropriate  graphical  displays  and  an  entry 
device  to  facilitate  Instructor  Interaction 
and  evaluation. 

SMARTTS  has  Identified  the  need  for  a  fast 
time  simulation  capability  to  assist  the 
Instructor  In  developing  exercises.  This 


full  capability  would  require  the  fast-time 
simulation  capability  as  well  as  other 
elements  of  the  exercise  development 
function.  In  the  preprototype  SMARTTS  this 
capability  was  given  secondary  Importance 
to  the  monitoring,  control,  and  briefing 
capabilities.  Nevertheless,  a  fast-  time 
simulation  capability  at  a  remote  terminal 
was  provided  on  SMARTTS,  partly  due  to  a 
particular  design  of  the  system  which  has 
two  parallel  complete  operating  systems 
with  associated  simulation  models  and  two 
complete  Instructor  consoles  (l.e.,  one  In 
a  classroom  and  the  other  In  an  attack 
center).  The  SMARTTS  system  permits  the 
fast- time  running  (l.e.,  at  a  rate  up  to 
16X)  of  any  exercise  scenario  in  the 
library.  All  of  the  tactically  relevant 
variables  and  performance  indicators  can  be 
evaluated  on  either  of  the  Instructor 
consoles.  Modifications  of  any  of  the 
tactical  parameters  can  be  effected  from 
these  consoles,  with  the  modified  exercises 
permanently  stored  for  later  training  use. 
This  SMARTTS  capability,  during  the  limited 
time  In  which  It  has  been  given  operational 
use,  has  been  found  to  greatly  assist  the 
instructors  In  preparing  and  evaluating 
scenario  exercises.  Requirements  had  been 
earlier  identified  to  more  fully  develop 
this  capability  in  the  later  developmental 
versions  of  SMARTTS;  initial  indications  to 
date  are  that  these  planned  additions 
should  be  carried  out. 


Instructor  Interface  Language.  The  typical 
instructor  In  military  training  systems  has 
a  good  operational  background  (e.g., 
submarine  operating  experience  for  a 
submarine  tactical  instructor),  although  a 
limited  instructional  and  computer 
background,  particularly  with  regard  to 
operating  particular  training  devices. 
Hence,  It  is  desired  to  have  a  computer 
language  developed  to  facilitate  the 
Instructor's  Interface  with  the  training 
device.  This  Interface  lanuage  should 
permit  the  Instructor  to  Interact  using 
somewhat  standardized  operational 
terminology.  Instructional  terminology, 
and/or  near-English  terminology.  It  should 
be  designed  to  accommodate  those  activities 
normally  performed  by  the  Instructor  when 
interfacing  to  the  training  device.  With 
regard  to  exercise  development,  for 
example,  this  language  should  accept 
dlscriptlve  commands  by  the  Instructor  in 
the  form  of  the  normal  operational 
parameters  (e.g.,  whereas  the  computer 
manipulates  targets  on  an  X,  Y,  Z  grid, 
operational  personnel  often  view  the 
geographic  situation  in  terms  of  range, 
bearing,  and  depth/evaluation).  Further¬ 
more,  the  interface  language  should  be 
structured  to  contain  a  large  set  of  macro 
functions  that  readily  translate  the 
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Instructor's  needs  Into  exercise, 
performance  Indicator,  cue,  feedback 
display,  etc.,  modifications.  For  example, 
the  capability  should  be  provided  to  enable 
the  Instructor  to  readily  tailor  feedback 
displays  to  particular  exercises.  This 
macro  function  should  enable  the  Instructor 
to  rapidly  provide  the  Information 
necessary  to  configure  the  display  In  terms 
as  close  as  possible  to  those  operationally 
used,  and  then  enable  the  computer  to 
automatically  configure  the  display.  This 
might  be  accomplished  using  a  digital 
Interface  drawing  tablet,  with  appropriate 
parameter  data  readily  accessible,  and  with 
standardized  manipulation  algorithms 
available. 

SMARTTS,  although  not  possessing 
capabilities  to  this  extent,  does  permit 
Instructor  flexibility  in  setting-up  and 
tailoring  trainee  information  displays. 
SMARTTS,  for  example,  provides  several 
generic  plot  formats  of  X  versus  Y  with 
which  the  Instructor  can  display  any 
parameter  (I.e.,  performance  Indicators  and 
tactical  variables)  as  a  function  of  any 
other  parameter.  The  Instructor  simply  has 
to  indicate  which  variables  he  wishes  to 
plot  and  they  would  be  automatically 
plotted.  The  Instructor  can  also  set-up 
several  groups  of  parameters  to  be 
called-up  for  display  simultaneously  by 
requesting  the  group(s)  rather  than  each 
individual  parameter.  These  capabilities 
are  not  only  useful  during  the  exercise 
generation  function  but  are  also  Intended 
for  the  briefing  function.  SMARTTS  has 
Identified  the  need  for  an  instructor 
Interface  language  for  exercise 
development,  but  has  not  formally  developed 
such  a  language  as  part  of  the  preprototype 
system,  although  some  capabilities  of  this 
type  are  Included  in  the  preprototype. 


MONITOR  AND  CONTROL  TRAINING.  The 
Instructional  support  suBsystem  should 
provide  a  predominence  of  its  capabilites 
for  the  monitoring  and  control  of  training, 
and  for  the  briefing  function  which  is 
addressed  later.  Suggested  monitoring  and 
control  capabilities  are  presented  below, 
In  reference  to  Figure  4. 


Exercise  set  up.  At  the  point  when  the 
scenario  exercise  will  be  run  on  the 
training  device  during  actual  training,  it 
should  have  undergone  prior  development  and 
evaluation.  Hence,  typically,  exercise  set 
up  would  be  a  matter  of  simply  selecting 
the  appropriate  exercise  from  the  library 
and  initiating  the  problem  on  the  training 
device.  Some  minor  modification  may  also 
be  desired  at  this  time.  An  exercise 
library,  therefore,  should  be  available  to 


the  Instructor  from  which  he  can  select  the 
appropriate  exercise.  Actual  set  up  on  the 
device  should  be  as  automated  as  possible, 
so  as  to  free  the  instructor  to  perform 
other  duties,  and  to  enable  quick  turn 
around  time  between  training  exercises  if 
desired.  The  exercise  set-up  procedure  on 
today's  training  devices,  even  after  the 
exercises  have  been  fully  developed.  Is 
often  cumbersome  and  time  consuming;  manual 
entry  of  all  the  set  up  parameters  is  often 
necessary  at  the  time  of  Initiating  the 
exercise  (e.g..  In  some  instances  this  may 
take  up  to  45  minutes).  This  laborious 
entry  process  should  be  unnecessary  for 
standard  exercises;  on  a  sophisticated 
training  device  the  exercise  parameters  can 
simply  be  stored  on  a  disk  for  automatic 
entry.  A  capability  should  be  provided  to 
enable  the  instructor  to  modify  any  of  the 
parameters  at  the  time  of  Initiation,  if  he 
so  desires. 

SMARTTS  provides  an  instructor's  console 
immediately  adjacent  to  the  fire  control 
system  consoles  from  which  the  instructor 
can  select  an  exercise  from  the  exercise 
library  and  have  it  automatically  entered 
upon  command.  He  can  also  investigate  and 
manipulate  many  of  the  relevant  tactical 
variables  at  this  time  to  modify  the 
exercise.  Furthermore,  if  the  instructor 
so  desires,  a  modified  copy  of  the  exercise 
can  be  automatically  stored  in  the  exercise 
library  as  a  new  exercise  for  later  use. 
This  type  of  capability  substantially 
reduces  the  low  level  time  consuming  task 
of  set-up. 


Exercise  Monitor.  The  instructor  must 
monitor  both  the  scenario  and  the  trainee 
activities.  Relevant  monitoring  informa¬ 
tion  should  be  generated  by  the  training 
device,  with  much  of  the  information 
selectable  by  the  instructor  in  real-time 
when  he  requires  it.  It  should  be  provided 
to  him  in  a  timely  manner  and  in  a  clearly 
understood  and  meaningful  format.  Cues 
should  be  provided  as  appropriate  to 
off-load  an  appropriate  part  of  his 
monitoring  function.  This  information 
should  be  provided  to  the  instructor  in  a 
convenient  location  which  enables  him  to 
perform  his  other  duties  with  a  minimum  of 
travel.  Both  alphanumeric  and  graphical 
information  should  be  provided,  depending 
on  the  purpose  of  the  information  and  the 
precision  the  instructor  requires. 

SMARTTS  provides  an  instructor's  console  in 
the  attack  center,  immediately  adjacent  to 
the  fire  control  party  and  the  MK117  fire 
control  system.  An  identical  Instructors 
console  is  located  in  the  classroom  for 
briefing  purposes.  SMARTTS  automatically 
records  a  wide  variety  of  tactical 
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variables  and  performance  Indicators.  This 
Information  Is  available  to  the  Instructor 
upon  request  at  the  Instructor's  console. 
Both  graphical  and  alphanumeric  formats  are 
provided,  depending  upon  the  Information 
and  the  Instructor's  desires.  The  display 
Is  automatically  up  dated  over  time  to 
provide  the  most  recent  Information,  as 
well  as  to  provide  historical  Information 
from  the  beginning  of  the  exercise  or  other 
designated  times.  Cues  are  provided  to  the 
Instructor  regarding  (1)  performance 
Indicator  values  or  tactical  parameter 
values  that  have  gone  beyond  predetermined 
limits  (e.g.,  probability  of  counterdetec¬ 
tion  going  beyond  50  percent);  and  (2) 
pending  action  by  the  automatic  Interactive 
target  (AIT)  (see  below  regarding  control 
capabilities).  The  display  formats  used  to 
provide  Information  to  the  Instructor  were 
developed  with  regard  to  the  requirements 
of  the  training  situation  and  the  type  of 
Information  submarine  officers  normally 
deal  with  (e.g.,  llne-of-slght  diagram). 
Hence,  the  Instructor  need  not  return  to 
the  program  operator  consoles  to  monitor 
exercise  progress.  Rather,  he  can  stay 
with  the  fire  control  party  to  monitor  the 
trainees'  actions,  and  still  have  the 
normal  scenario  monitoring  Information 
available.  Furthermore,  SMARTTS  provides 
the  Instructor  with  a  variety  of  additional 
Information  concerning  trainee  performance 
and  the  problem  status. 


Data  Recording.  The  analysis  of  instructor 
functions  and  tasks  revealed  that  typical 
Instructors  spend  a  relatively  large  amount 
of  time  recording  exercise  data,  if  they 
provide  this  type  of  feedback  to  the 
trainees  after  the  exercise.  Furthermore, 
the  better  postscenario  briefing  sessions 
are  those  In  which  the  Instructor  has 
substantial  Information  available  for 
presentation  and  discussion  with  the 
trainees  regarding  various  aspects  of  the 
problem.  Much  of  the  Information  recorded 
by  the  Instructor  Is  normally  generated  by 
the  training  device  (e.g.,  tactical 
parameters  and  events).  The  typical 
computer-based  training  device  has  the 
capability  to  record  these  data  for  later 
accessing  by  the  Instructor  during  the 
postscenario  briefing.  In  addition,  a 
considerable  part  of  the  Instructor's  tasks 
Is  to  monitor  and  record  actions  of  the 
trainee  that  could  not  be  automatically 
recorded  by  the  system  (e.g.,  communication 
between  team  members,  coordination).  These 
types  of  subjective  performance  Indicators 
would  be  relatively  difficult  to  achieve 
via  automated  means  In  many  training 
situations.  Hence,  the  instructor  should 
focus  a  substantial  portion  of  his  time 
monitoring  and  recording  these  types  of 
performance  Indicators  for  later  feedback. 


The  training  device  could  assist  the 
Instructor  by  facilitating  his  recording  of 
this  Information  via  a  standardized 
Interface  with  th4  training  device. 

The  SMARTTS  system  records  a  wide  variety 
of  tactical  variables  that  occur  during  the 
exercise  scenario.  Additionally,  a  set  of 
objective  performance  Indicators  are 
generated  and  also  automatically  recorded 
during  the  exercise  scenario.  Finally,  a 
capability  Is  provided  for  the  instructor 
to  enter  a  variety  of  subjective 
observations  into  the  training  devlce/- 
computer  storage  facility.  For  example.  If 
the  Instructor  observes  highly  proficient 
conmunlcatlon  between  the  fire  control 
coordinator  and  the  plot  coordinator  during 
a  particular  time  of  the  scenario  regarding 
a  particular  aspect  of  the  scenario,  he  can 
easily  enter  this  observation  into  the 
computer  by  designating  the  appropriate 
observation.  All  of  the  above  data  are 
readily  accessible  to  the  instructor  during 
the  exercise,  and  also  following  the 
exercise  for  discussion  In  the  briefing 
session.  This  set  of  capabilities  not  only 
reduces  the  Instructor's  load,  but  also 
provides  tools  to  the  Instructor  for 
training  that  he  here-to-fore  did  not 
have.  For  example,  the  Instructor 
typically  was  unable  to  monitor  changes  in 
the  fire  control  system  solution  accuracy 
over  the  course  of  the  exercise  as  a 
function  of  ownshlp's  maneuvering;  this 
information  Is  now  automatically  recorded 
and  can  be  accessed  by  the  instructor  at 
any  time.  Including  during  the  postscenario 
briefing  session. 


Scenario/Exercise  Control.  The  instructor 
should  be  provided  with  a  wide  range  of 
control  capabilities  to  enable  him  to 
readily  control  all  aspects  of  the 
scenario,  and  also  control  presentation  of 
pertinent  Information  to  the  trainees  as 
necessary.  The  control  capabilites  should 
be  provided  In  a  convenient  location, 
similar  to  that  for  the  monitoring 
Information,  preferably  near  the  trainees 
such  that  the  Instructor  can  continually 
monitor  the  trainees  and  the  problem  while 
entering  necessary  control  commands. 
Ideally,  the  instructor  would  spend  a 
minimum  of  time  controlling  the  exercise, 
devoting  most  of  his  resources  to  other 
training-related  activities.  In  this 
regard,  scripted  scenarios  should  be 
available  wherein  the  target  has  predeter¬ 
mine  actions.  Additionally,  other  types  of 
control  for  the  target  and  other  aspects  of 
the  problem  would  be  desirable  If  they 
freed  the  Instructor  from  these  tasks.  It 
should  be  noted,  however,  that  the 
instructor  should  be  allowed  to  control  any 
of  the  aspects  he  so  desires.  In  addition 


to  the  scenario  control  capabilities,  the 
information  presentation  capabilities 
should  likewise  be  located  for  appropriate 
viewing  by  the  trainees,  and  so  that  the 
instructor  can  control  the  information 
presentation  via  convenient  and  effective 
media.  The  location  of  this  information 
presentation  (e.g.,  feedback)  would 
normally  be  either  in  the  operational 
simulator  environment  (e.g.,  attack  center) 
and/or  in  a  briefing  room.  In  either  case, 
the  instructor  should  have  complete  control 
over  the  information  presented,  enabling 
him  rapid  configuration  of  the  desired 
displays. 

The  instructor's  console  in  the  SMARTTS 
system  is  located  near  the  fire  control 
party  in  the  attack  center.  The  instructor 
is  presented  with  monitoring  information  on 
a  color  graphic  CRT  as  well  as  the  plasma 
display  entry  device.  The  instructor  can 
control  many  of  the  relevant  tactical 
problem  variables  (e.g.,  maneuver  targets) 
from  this  console,  as  well  as  control  a 
wide  variety  of  information  to  be  presented 
to  the  trainees.  The  information 
presentation  to  the  trainees  is  achieved  in 
the  attack  center  via  two  overhead-mounted 
color  CRT  displays.  A  variety  of  display 
formats  can  be  put  up  independently  on  each 
of  these  CRTs  via  command  from  the 
instructor's  console.  The  similar 
instructor's  console  located  in  the 
adjacent  classroom  controls  the  information 
to  be  displayed  on  a  large  screen  display 
during  briefing  sessions. 

SMARTTS  has  the  normal  scripted  scenario 
capability,  along  with  an  automatic 
interactive  target  (AIT).  The  AIT 
capability  provides  a  computer-controlled 
model  of  an  enemy  target  platform.  This 
model  is  based  on  the  best  available 
intelligence  Information  for  the  particular 
platform.  The  model  automatically  controls 
the  target  in  response  to  the  evolving 
situation  events  in  real-time.  Cues  are 
provided  to  the  instructor  prior  to  any 
action  on  the  part  of  the  AIT,  enabling  the 
instructor  to  (1)  allow  the  AIT  to  carry 
out  its  planned  action,  (2)  override  the 
AIT's  planned  action  by  having  it  continue 
doing  it  current  activity,  (3)  enter  a 
different  target  course  of  action,  or  (4) 
take  over  manual  control  of  the  target. 
The  AIT  capability  is  intended  to  reduce 
the  instructor's  load  in  controlling  the 
principle  target,  which  has  been  observed 
to  be  considerable.  The  AIT,  furthermore, 
brings  in  the  best  available  intelligence 
information  to  orobabil istically  control  the 
target  in  a  realistic  fashion.  This 
capability  will  assist  in  off-setting 
experience  differences  and  biases  between 
instructors. 


Information  Presentation.  Information 
presentation  to  the  trainees,  in  the  form 
of  a  preproblem  briefing  or  postproblem 
feedback,  is  an  essential  part  of  an 
effective  training  process.  Appropriate 
information  must  be  generated  by  the 
training  system,  and  presented  to  the 
trainees  in  an  effective  way  to  assist 
their  assimilation.  A  wide  variety  of 
media  is  available  for  presenting 
information  to  the  trainees.  The  most 
common  is  that  of  verbal  presentation  by 
the  instructor.  Unfortunately,  verbal 
presentation  is  limited  with  regard  to 
handling  complex  relationships.  The  use  of 
augmenting  graphical  information  (i.e., 
pictures)  and  examples  has  been  shown  to 
substantially  increase  the  effectiveness  of 
training  (Lesgold,  Pellegrino,  Fokkema,  and 
Glaser,  1978).  Hence,  a  visual  information 
display  capability  should  be  made  available 
in  the  instructional  subsystem,  particu¬ 
larly  for  complex  training  situations.  A 
variety  of  capabilities  may  be  associated 
with  that  information  display,  such  as  a 
fast  time  model  to  explore  alternative 
actions  in  a  given  problem.  Also,  a 
variety  of  information  should  be  available 
in  the  training  device  regarding  the 
recently  completed  exercise  scenario  for 
presentation  to  the  trainees.  Ready  access 
to,  and  flexible  control  over,  this 
information  and  these  capabilities  should 
be  provided  to  the  instructor  at  an 
appropriate  location.  Obviously,  the 
monitor,  record,  and  control  capabilities 
act  together  with  the  information  display 
capabilities  to  provide  appropriate 
information  to  the  trainees.  The 
information  presentation  capabilities  are 
further  discussed  below  under  the  briefing 
function  which  overlaps  considerably  with 
this  monitor  and  control  function. 

The  monitor  and  control  capabilities  of  the 
instructional  subsystem  are  quite  extensive 
on  SMARTTS,  as  part  of  the  submarine  combat 
system  trainer.  These  capabilities  are 
generic,  although  their  specific  character¬ 
istics  have  been  tailored  to  the  submarine 
tactics  training  problem.  The  above 
discussion  presents  a  very  general  summary 
overview  of  the  particular  characteristics 
incorporated  in  SMARTTS  for  this  instructor 
function,  and  its  associated  tasks,  issues 
and  considerations. 


BRIEFING.  The  briefing  of  trainees  is  a 
major  training  process  function  of  the 
instructor.  It  obviously  overlaps  with  the 
monitor  and  control  tasks  as  noted  above, 
and  also  overlaps  with  the  trainee 
interface  requirements  of  the  instructional 
subsystem.  It  is  the  opinion  of  this 
author  that  many  of  the  trainee  Interface 
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characteristics  of  the  instructional 
subsystem  are  the  same  as  those  required 
for  the  briefing  function.  The  briefing 
function  of  the  instructor  deals  with 
providing  external  information  to  the 
trainees  to  correlate  with  their  actual  or 
desired  actions.  This  information  is  a 
primary  means  by  which  training  actually 
occurs.  For  example,  this  information 
provides  feedback  informing  the  trainee  of 
how  successful  his  particular  actions  were 
with  regard  to  the  operational  objectives. 
Since  a  major  objective  of  training  is  to 
develop  an  awareness  on  the  part  of  the 
trainee  of  the  relationship  between 
available  information,  alternative  actions 
that  can  be  taken,  and  the  resultant  likely 
situation  outcomes,  external  information 
presented  to  the  trainee  is  extremely 
important.  Hence,  the  effectiveness  with 
which  information  can  be  presented  to  and 
assimulated  by  the  trainee  will  have  a 
direct  bearing  on  the  effectiveness  of  the 
training  process.  Information  presented  to 
the  trainee  attempts  to  achieve  the 
following: 

(1)  explain  a  particular  evolving 
situation  (e.g.,  events  that  occur 
during  a  tactical  encounter), 

(2)  identify  possible  alternative 
situations  and  actions  (e.g.,  the 
types  of  targets  likely  to  be 
encountered  in  this  area,  each 
target's  likely  set  of  tactics,  and 
appropriate  defenses), 

(3)  the  outcome  of  the  trainee's 
particular  actions  during  the 
problem  (e.g.,  the  trainee  teams 
tactical  approach  on  a  target), 

(4)  the  relationships  between  tactical 
variables  that  were  relevant  in  the 
problem  (e.g.,  the  relationship 
between  ownship  speed  and 
probability  of  counterdetection),  and 

(5)  the  likely  impact  of  various 
alternative  trainee  actions  on  the 
situation  outcome  (e.g.,  the  impact 
of  different  ownship  speeds  on 
achieving  a  successful  target 
approach).  These  catergories  of 
information  might  be  differentially 
brought  up  prior  to  a  real-time 
training  exercise,  during  a  freeze 
pause  in  that  exercise,  or  following 
the  exercise. 

The  information  categories  fall  into  two 
general  classes.  The  first  is  to  provide 
Information  regarding  a  particular 
problem.  This  is  typically  in  the  form  of 
feedback  after  the  problem  occurred.  It 
may  Involve  dissecting  the  problem  to 


investigate  the  trainee's  actions,  why  he 
took  those  actions,  what  options  were 
available  to  him,  and  so  on.  The  second 
class  of  information  is  somewhat 
independent  of  the  particular  actions  on 
the  part  of  the  trainee.  Rather,  it  deals 
with  the  general  set  of  possible  problems, 
investigating  relevant  issues  surrounding  a 
particular  set  of  problems.  This  would 
normally  occur  in  a  briefing  session  prior 
to  the  real-time  exercise  on  the  device. 
It  may  also  evolve  during  the  postproblem 
briefing  session.  Whereas  the  former  class 
of  information  presentation  class  relies  on 
the  data  generated  and  collected  during  the 
training  exercise,  this  latter  relies  on  a 
fast  time  modeling  capability  to  generate 
the  various  alternative  actions,  tactical 
parameters,  and  performance  indicators. 
This  briefing  capability  should  be  located 
where  most  convenient  to  the  trainees,  and 
conducive  to  the  types  of  activities 
occurring  during  the  training  process. 

Information  displays  in  the  SMARTTS  system 
have  been  provided  both  in  the  attack 
center  and  in  the  classroom.  These  two 
locations  recognize  the  different 
activities  that  may  take  place  in  both. 
The  classroom  can  operate  in  conjunction 
with  the  attack  center  (e.g.,  monitoring 
the  ongoing  exercise  in  the  attack  center), 
or  both  can  operate  independently  on 
separate  problems.  A  variety  of 
performance  indicators  are  available  for 
display  to  provide  the  trainees  with 
information  concerning  the  various  aspects 
of  their  performance.  The  intention  in 
SMARTTS  has  been  not  to  develop  performance 
indicators  pertaining  to  good  or  bad 
performance,  but  rather  to  generate 
information  that  can  be  meaningful  from  the 
standpoint  of  informing  the  trainee  about 
the  scenario  and  his  performance.  Often, 
performance  indicators  change  in  an 
opposing  fashion,  necessitating  trade-offs 
between  them  when  selecting  the  appropriate 
tactical  action.  From  a  training  process 
standpoint,  the  important  consideration  is 
not  how  well  the  trainee  did,  but  rather 
that  the  trainee  understand  the  impact  of 
his  actions  ai  i  the  other  actions  available 
to  him.  SMARTTS  provides  the  instructor 
with  a  wide  range  of  capabilities  to  access 
the  various  information  that  was  recorded 
during  the  training  problem.  A  variety  of 
information  display  formats  are  available 
for  presenting  and  discussing  this 
information,  as  well  as  a  range  of 
flexibility  to  dissect  the  information  and 
focus  on  particular  aspects  of  the 
problem.  The  instructor's  subjective 
observations,  which  were  entered  during  the 
exercise,  are  also  available  for  a 
reference  during  the  postproblem  briefing 
session.  Also,  time-flag  units  are 
available  at  several  trainee  locations  to 
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enable  the  trainees  to  place  a  time-tagged 
indication  in  the  scenario  recording  at  a 
point  when  they  were  concerned  with  a 
particular  issue  and  unable  to  bring  it  to 
the  intention  of  the  instructor.  These 
trainee-instituted  time-tags  can  be 
accessed  during  the  postproblem  briefing 
session  for  investigation. 

The  fast-time  simulation  capability, 
discussed  above  several  times,  is  also 
available  in  SMARTTS  to  generate  problems 
for  presentation  and  discussion.  The 
fast-time  capability  may  be  used  in  a 
preexercise  briefing  wherein  the  trainees 
can  be  given  a  preview  of  the  particular 
exercise  they  will  encounter,  or  other 
relevant  exercises.  Various  performance 
indicators  and  tactical  parameters  can  be 
Investigated  at  this  prebriefing  time  to 
focus  on  the  aspects  of  importance  during 
the  problem.  Alternative  sets  of  ownship 
and/or  target  actions  can  also  be  addressed 
at  this  time  to  explore  their  impact  on  the 
various  tactical  parameters  and  performance 
indicators.  This  capability  is  also 
available  during  a  problem  freeze  and 
during  the  postproblem  briefing  session. 
During  the  latter  session,  alternative  sets 
of  actions  may  be  explored  for  the 
particular  problem  just  completed  in  the 
attack  center.  It  is  important  to  note 
that  models  are  provided  in  SMARTTS  to 
generate  information  that  would  normally  be 
generated  by  the  trainee  team  (e.g.,  target 
motion  analysis  solution).  This  informa¬ 
tion,  generated  in  fast  time,  is  useful  for 
comparison  with  the  actual  values  generated 
by  the  trainee  team  with  regard  to  the 
alternative  actions  being  investigated  for 
comparative  purposes. 

It  is  important  that  an  adequate  human 
factors  design  be  implemented  for  the 
instructor/computer  interface  with  regard 
to  the  briefing  capabilities.  At  this 
time,  the  instructor  is  standing  on-stage 
and  has  to  perform  in  a  timely  fashion. 
These  briefing  capabilites  provide  the 
instructor  with  a  substantial  amount  of 
Information  to  access  and  present  to  the 
trainees.  However,  it  must  be  facilitated 
in  such  a  way  that  it  can  be  brought  up 
flexibly  and  quickly,  with  relative  ease  on 
the  part  of  the  instructor.  It  must  also 
be  presented  clearly  to  the  trainees.  The 
design  of  the  SMARTTS  instructor's  console 
(e.g.,  a  color  graphic  CRT  and  a  plasma 
display  entry  device),  as  noted  above,  has 
been  designed  to  facilitate  this  interface. 


Training  System  Management 

A  variety  of  capabilities  are  desirable  for 
training  system  managment.  The  most 
Important  capability  that  the  device 


automation  can  assist  is  the  recording  and 
storage  of  data  generated  during  exercise 
scenario  runs  and  the  statistical  analysis 
routines  to  enable  investigation  of  these 
data  at  a  later  time.  As  noted  earlier, 
important  training  system-related  data  are 
generated  each  time  the  training  device  is 
used.  These  data  can  be  extremely  useful 
in  evaluating  and  upgrading  a  variety  of 
aspects  of  the  training  process  (e.g., 
development  of  exercises  as  noted  above). 
The  SMARTTS  front-end  analysis  has 
identified  the  need  for  many  of  these 
capabilities.  The  preprototype  SMARTTS 
system  has  the  capability  to  record  and 
long-term  store  exercise  data  for  later 
analysis.  Additional  analysis  capabilities 
should  become  a  part  of  the  later  SMARTTS 
developmental  units. 


SUMMARY 

The  training  device  is  more  than  a 
simulator.  It  has  both  simulation  and 
instructional  subsystems,  each  specifically 
designed  for  a  vastly  different  purpose. 
The  instructional  subsystem  encompasses 
capabilities  that  directly  support  the 
training  process.  These  capabilities  focus 
on  the  instructor,  the  trainee,  and 
training  system  management.  Their 
incorporation  in  modern  training 
devices/training  systems  is  essential  to 
achieve  the  high  levels  of  training 
cost/effectiveness  requisite  for  adequate 
operational  effectiveness  of  modern  weapon 
systems. 

Although  many  training  support  capabilities 
should  be  automated,  such  as  the  computer 
driven  automatic  interactive  target  and 
others  discussed  in  this  paper,  many 
additional  capabilities  that  are  part  of 
the  instructional  subsystem  can  be 
effectively  accomplished  via  traditional 
manual  means  (e.g.,  instructor  guidelines 
for  exercise  development).  The  training 
device/training  system  should  provide  a 
range  of  capabilities  in  the  instructional 
subsystem  selected  on  the  basis  of  their 
cost/effectiveness  impact  on  the  training 
process.  These  capabilities,  furthermore, 
should  be  flexibly  tailored  to  each 
specific  training  situation.  The  training 
device/training  system  can  greatly  assist 
the  instructor  in  enabling  him  access  to 
here-to-fore  unavailable  complex 
information  directly  impacting  trainee 
performance  (e.g.,  probability  of 
counterdetection).  Presentation  of  this 
complex  information  can  also  be  enhanced 
via  presentation  on  highly  effective  media 
(e.g.,  large  screen  graphical  display).  A 
variety  of  data  manipulation  capabilities 
can  enable  disecting  various  operational 
problems,  investigation  of  alternative 
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actions.  Illustration  of  a  range  of 
examples,  and  so  on  (e.g.,  via  fast-time 
computer  modeling).  These  and  many  other 
computer-based  and  manual  capabilities  make 
up  essential  elements  of  the  Instructional 
subsystem. 

The  Instructional  subsystem  encompasses 
elements  beyond  those  discussed  in  this 
paper,  such  as  a  variety  of  manual  media 
typically  used  in  the  training  process. 
The  instructor  himself,  for  example,  is 
probably  the  single  most  important  element 
of  the  instructional  subsystem  and  the 
training  system  (Gardenier  and  Hanmell, 
1981).  Empirical  research  has  shown  that 
the  instructor  can  have  a  substantial 
impact  on  the  effectiveness  of 
simulator-based  training.  The  capabilities 
discussed  in  this  paper  provide  tools  for 
the  instructor  to  enable  him  to  achieve  a 
more  effective  training  process. 
Nevertheless,  the  fundamental 
characteristics  of  the  instructor  himself, 
independent  of  these  tools,  may  have  a 
substantial  impact  on  the  effectiveness  of 
any  training  situation. 

As  discussed  early  in  this  paper,  we  are 
now  emerging  on  the  third  generation 
training  device,  emphasizing  the  importance 
of  the  instructional  subsystem  and  built-in 
training  technology  capabilities  to  enhance 
the  training  process.  Although  the 
emphasis  given  to  the  instructional 
subsystem  in  the  past  has  been  relatively 
minor,  its  importance  is  becoming 
recognized,  with  increasing  emphasis  given 
to  these  capabilities  in  the  design  of 
today's  training  devices/training  systems. 
SMARTTS  is  one  example,  and  a  forerunner  of 
that  trend.  Another  example  Is  a  set  of 
guidelines  for  deck  officer  training 
systems  (Gynther,  Hammell,  Grasso,  and 
Pittsley,  1982)  which  address  the  use  and 
design  of  the  shiphandling/ship  bridge 
simulator  for  training  senior  commercial 
ship  deck  officers.  This  report  recognizes 
and  places  emphasis  on  "three  major 
elements  of  the  training  system  -  the 
simulator  design,  the  training  program 
structure,  and  the  Instructor 
qualifications"  (page  1).  This  document, 
which  Is  Intended  to  provide  training 
system  design  guidance  to  the  potential 
users  of  shiphandling/ship  bridge 
simulator-based  training  devices,  addresses 
and  recommends  many  of  the  training 
assistance  characteristics  discussed  In 
this  paper,  and  embodied  in  the  SMARTTS 
preprototype. 

The  instructional  subsystem  has  been 
presented  in  this  paper  as  one  of  two  major 
parts  of  the  training  device.  As  such  the 
training-related  capabilities  were  strongly 
emphasized,  while  the  fundamental 


engineering  aspects  of  this  subsystem  were 
played  down.  In  the  past,  along  with  the 
lack  of  adequate  Instructional 
capabilities,  most  training  devices 
received  inadequate  attention  and  scrutiny 
of  their  training-related  capabilities. 
Following  from  the  effective  procedures 
established  to  monitor  the  design  and 
development  of  the  hardware  and  software 
aspects  of  the  training  device,  a  similar 
procedural  policy  should  be  established  by 
training  device  procurement  activities  to 
monitor  and  evaluate  the  potential  training 
effectiveness  of  the  training  device. 
These  procedures  should  evaluate  the 
training  device  from  its  initial  conceptual 
design  stages  on  through  development, 
factory  acceptance  testing,  and  site 
acceptance  testing.  The  focus  of  these 
additional  evaluations  should  not  be  on  the 
specific  hardware  and  software  engineering 
aspects,  but  rather  on  the  potential  impact 
of  the  trainers  and  their  development 
characteristics  on  training  effectiveness. 
The  evaluation  of  the  potential  training 
device's  characteristics  should  not  end  with 
the  initial  conceptual  design;  rather,  it 
should  continue  to  monitor  the  specific 
details  of  the  implementation  of  those 
initial  concepts.  For  example,  a  detailed 
human  factors  review  of  the  various 
characteristics  of  the  instructor  interface 
should  be  accomplished  at  appropriate 
stages  during  the  development  process  to 
verify  effective  design  details.  SMARTTS 
developed  a  training  system  evaluation  test 
(TSET)  to  be  conducted  at  the  time  of  the 
Factory  Acceptance  Test  for  the  hardware 
and  software.  The  TSET  addressed  specific 
characteristics  of  the  design  as  they  were 
expected  to  impact  the  effectiveness  of  the 
training  process.  This  type  of  evaluation 
should  take  place  at  all  stages  from 
initial  conceptual  design  through  the 
production  run  of  a  particular  device,  and 
continue  after  the  device  is  operational  to 
constantly  evolve  its  capabilities  and 
hence  improve  its  training  effectiveness. 

This  paper  focused  heavily  on  training 
system  aids  for  assisting  the  instructor, 
and  providing  an  effective  Interface  with 
the  trainee.  It  did  address,  although  to  a 
limited  extent,  the  importance  of  the 
training  methodologies  employed  for 
achievement  of  an  effective  training 
process.  The  instructor,  the  training 
device  and  its  tools  are  mechanisms  for 
implementing  an  effective  training 
process.  The  methods  they  use  in 
Implementing  this  process  are  extremely 
important  for  achieving  maximally 
cost/effective  training.  The  training 
methods  are,  obviously,  an  extremely 
important  part  of  the  instructional 
subsystem.  They  represent  the  strategy 
that  is  employed  to  achieve  the  training 


145 


objectives.  As  training  system  capabllltes 
Improve,  the  breath  and  potential 
effectiveness  of  available  training 
methodologies  also  increase.  For  example, 
learning  by  example  has  been  shown  to  be  an 
effective  training  method  (Lesgold  et  al, 
1978).  This  can  be  accomplished  in  the 
modern  training  system  in  a  variety  of 
ways.  Including  several  facilitated  by 
capabilities  discussed  In  this  paper  (e.g., 
fast-time  modeling  in  the  classroom  to 
investigate  alternative  tactics).  By 
providing  appropriate  instructional  support 
capabilities  on  the  training  device, 
effective  training  methods  that  could  not 
be  used  due  to  the  problem  complexity  or 
other  situation  limitations  are  now 
available. 

The  training  expert  must  also  be  careful 
not  to  overlook  other  training  methodology 
principles  which  are  available  but  might  be 
overshadowed  by  the  automated  capabilities 
provided  on  sophisticated  training 
devices.  For  example,  overlearning  is  a 
potentially  effective  approach  for  the 
training  of  individuals  to  perform  under 
high  stress  conditions;  to  take  advantage 
of  the  benefits  of  overlearning  in  such 
situations  does  not  require  any  particular 
instructional  subsystem  capabilities 
(Fitts,  1965).  As  always,  the  most 
cost/effective  training  process  will  be 
achieved  by  taking  into  account  all  of  the 
relevant  elements  that  impact  that  process, 
including  the  simulation  characteristics  of 
the  training  device,  the  training  methods 
employed,  and  the  capabilities  of  the 
instructional  subsystem. 


REFERENCES 

Lesgold,  A.M.,  J.W.  Pelligrino,  S.O. 
Fokkema,  and  R.  Glaser.  "Cognitive 
Psychology  and  Instruction"  in  Proceedings 
of  Data  International  Conference  held  at 
the  Free  University  of  Amsterdam,  June 
13-17,  1977  sponsored  by  the  NATO  Special 
Program  Panel  on  Human  Factors.  New  York, 
Plenum  Press,  1978. 

Chenzoff,  A.P.  et  al.  "Guidelines  For  - 
Training  Situation  Analysis  (TSA)." 
Applied  Science  Associates,  Incorporated, 
Valencia,  Pennsylvania,  July  1965. 

Comptroller  General.  Report  to  the 
Congress  of  the  United  States; 
"Effectiveness  Of  U.S.  Forces  Can  Be 
Increased  Through  Improved  Weapon  System 
Design."  U.S.  General  Accounting  Office, 
PSAD-81-17,  January  29,  1981. 

Krahenbuhl,  G.S.,  P.W.  Darst,  J.R.  Marett, 
L.C.  Reuther,  and  S.H.  Constable. 


"Undergraduate  Pilot  Training:  Instructor 
Pilot  Behavior  And  Student  Stress  And 
Performance,  Level  II."  Department  of 
Health  and  Physical  Education,  Tempe, 
Arizona,  July  1980. 

Charles,  J.P.  "Instructor  Pilot's  Role  In 
Simulator  Training  (Phase  III)."  U.S. 
Naval  Training  Equipment  Report, 
NAVTRAEQUIPCEN  76-C-0034-2,  June  1978. 

Gardenier,  J.S.  and  T.J.  Hammell. 
“Shiphandling  Simulator  Standards  and 
Training  Technology."  U.S.  Coast  Guard, 
Washington,  D.C.  and  CAORF/Ship  Analytics, 
North  Stonington,  Connecticut.  Paper 
presented  at  the  Second  International 
Maritime  Simulation  Conference,  Kings 
Point,  New  York,  June  1981. 

Gynther,  J.W.,  T.J.  Hammell,  J.A.  Grasso, 
and  V.M.  Pittsley.  "Simulators  For  Mariner 
Training  And  Licensing:  Guidelines  For 
Deck  Officer  Training  System."  National 
Maritime  Research  Center,  Kings  Point,  New 
York,  January  1982. 

NAVEDTRA  106A.  "Interservice  Procedures 
For  Instructional  Systems  Development." 
Center  for  Education  Technology,  Florida 
State  University  and  U.S.  Army  Combat  Arms 
Training  Board,  Ft.  Benning,  Georgia,  1975. 

Fitts,  P.M.  "Factors  In  Complex  Skill 
Training"  in  R.  Glaser  (Ed.)  "Training 
Research  and  Education."  Wiley,  New  York, 
1965. 

Hammell,  T.J.  "The  Training  Device  Is  More 
Than  A  Simulator."  Ship  Analytics, 
Incorporated,  North  Stonington, 
Connecticut.  Paper  presented  at  the  Second 
International  Maritime  Simulation 
Conference,  Kings  Point,  New  York,  June 
1981. 

Hammell,  T.J.,  J.W.  Gynther,  J.A.  Grasso 
and  M.E.  Gaffney.  "Simulators  For  Mariner 
Training  and  Licensing  Phase  2: 
Investigation  Of  Simulator  Characteristics 
For  Training  Senior  Mariners."  National 
Maritime  Research  Center,  Kings  Point,  New 
York,  January  1981. 

Hamnell,  T.J.  and  T.N.  Crosby. 
"Development  of  SMARTTS  Training 
Technology."  Eclectech  Associates,  North 
Stonington,  Connecticut  and  Naval  Training 
Equipment  Center,  Orlando,  Florida.  Paper 
presented  at  the  Second  Interservice 
Industry  Training  Equipment  Conference, 
Salt  Lake  City,  November  1980. 

Caro,  P.W.  "Aircraft  Simulators  and  Pilot 
Training"  in  Human  Factors,  1973. 


Hammell,  T.J.,  F.P.  Sroka,  and  F.L.  Allen. 
"Volume  I  of  II,  Study  of  Training  Device 
Needs  for  Meeting  Basic  Officer  Tactics 
Training  Requirements."  Naval  Training 
Device  Center,  Orlando,  Florida,  1971. 

Hammell,  T.J.,  C.E.  Gasteyer  and  A.J. 
Pesch.  "Volume  I  of  II,  Advanced  Officer 
Tactics  Training  Device  Needs  and 
Performance  Measurement  Technique." 
General  Dynamics/Electric  Boat  Division, 
Groton,  Connecticut,  November  1973. 


147/148 


PROCEDURES  MONITORING  AND  SCORING  IN  A  SIMULATOR 

Steve  Seidensticker 
Terry  Kryvay 

Tactical  and  Training  Systeme  Division 
Logicon,  Inc. 

San  Diego,  CA 


ABSTRACT 

As  the  quality  of  cockpit  simulation  increases  and  as  economic  pressures 
force  more  training  into  simulators,  the  number  and  type  of  training  tasks 
handled  in  the  simulator  are  also  expanding.  Many  of  these  new  tasks  are 
procedurea  oriented.  This  is  particularly  evident  in  pilot  replacement  training 
in  high  performance  aircraft.  In  such  situations  the  simulator  is  used  to 
expose  the  student  to  emergencies  and  other  situations  where  judgement  and 
the  ability  to  follow  certain  procedures  accurately  and  quickly  are  vital. 
To  do  this  rational  techniques  are  needed  to  detect  and  assign  meaning  to 
the  procedural  events.  This  paper  will  relate  in  some  depth  the  efforts 
to  incorporate  a  comprehensive  procedures  monitoring  and  scoring  facility 
in  an  Operational  Flight  Trainer. 


BACKGROUND 

The  quality  of  flight  and  weapons  systems 
simulation  in  military  simulators  has  advanced 
rapidly  with  the  application  of  digital  com¬ 
puters.  This  is  particularly  visible  in 
the  areas  of  visual  scene  and  weapons  system 
simulations.  Transitional  (from  one  aircraft 
to  another)  training  curricula  have  taken 
advantage  of  these  recent  advancements  and 
flight  trainers  are  now  being  used  in  scenarios 
where  more  than  just  flying  skills  and  the 
rote  responses  to  emergencies  are  being  taught. 
Instructors  are  now  subjecting  the  student 
to  situations  where  pre-planning,  headwork 
and  a  thorough  knowledge  of  the  aircraft 
and  its  systems  are  required. 

Ssnwsl  Eiafelffla 

The  quality  of  training  received  from 
theae  expensive  systems  depends  a  great  deal 
on  the  instructor' a  ability  to  operate  these 
trainers  and  of  his  ability  to  keep  up  with 
activities  in  the  cockpit.  Often  the  existence 
of  such  ability  is  an  exception  rather  than 
the  rule.  Thia  is  not  because  instructors 
are  incapable  of  the  tasks,  but  because  the 
complexities  of  operation  require  substantial 
training  and  experience  in  order  for  the 
instructors  to  become  proficient.  And  operation 
of  the  trainer  is  only  one  of  the  many  respons¬ 
ibilities  of  the  flight  instructor. 

An  example  of  such  a  situation  stems 
from  a  recent  improvement  to  the  Navy's  F-14 
Operational  Flight  Trainer  (OFT),  device 
2F95.  The  visual  system  has  been  enhanced 
to  simulate  carrier  operations  including 
catapult  launches.  The  training  syllabus 
has  taken  advantage  of  this  and  now  single 
engine  failures  are  being  practiced  during 
the  catapult  shots.  A  timing  problem  exists 


in  the  insertion  of  the  engine  failure  after 
the  catapult  stroke  has  started.  If  it  is 
not  inserted  at  precisely  the  correct  moment, 
the  simulation  is  not  accomplished  correctly 
and  the  training  value  is  lost.  Therefore 
the  instructor  spends  most  of  his  attention 
in  the  operation  of  the  trainer  and  not  to 
the  student's  reaction  to  the  emergency.  If 
the  student  crashes,  the  instructor  obviously 
notices  the  result,  but  does  not  know  what, 
if  anything,  the  student  was  doing  prior 
to  getting  wet. 

Recent  emphasis  has  been  placed  on 
simulator  deaign  to  help  the  instructor  with 
scenario  generation  and  problem  control.  One 
of  the  important  aspects  of  problem  control 
is  that  of  procedurea  monitoring  and  performance 
measurement.  Though  simulators  in  the  past 
have  attempted  to  provide  the  instructor 
with  this  type  of  information,  in  most  cases 
these  enhancements  are  the  first  to  be  ignored 
by  the  instructor. 

Early  Attempts 

An  example  of  recent  performance  measurement 
capabilities  are  on  the  existing  F-14  OFT. 
This  trainer  has  the  capability  of  monitoring 
the  performance  of  a  trainee  and  issuing 
a  hard-copy  printout  whenever  performance 
limits  are  exceeded.  The  instructor  may 
select  five  of  ten  flight  parameters  and 
set  a  low  and  high  limit  on  these  values. 
Be  may  also  segment  the  exercise  by  time. 
Then  by  paging  into  an  alphanumeric  display, 
he  may  observe  the  trainee  performance  with 
respect  to  these  flight  parameters. 

Some  of  the  problems  voiced  by  instructors 
with  respect  to  this  particular  mechanism 
are: 
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The  performance  monitoring  is  restricted 
to  flight  parameters  only. 

The  high  and  low  limits  are  not  standardized 
and  the  values  are  often  meaningless 
because  they  usually  must  be  related 
to  the  scenario. 

Indexing  by  time  into  the  training 
session  is  not  effective  in  that  instructors 
do  not  keep  track  of  a  training  session 
by  timet  but  by  training  task.  Time 
references  may  be  meaningful  if  used 
within  a  particular  training  task  but 
not  if  used  with  respect  to  the  entire 
training  session  (e.g.  two  minutes 
after  commencing  an  approach  is  meaningful, 
however,  17  minutes  after  commencing 
the  exercise  is  not). 

Setting  up  the  performance  measurement 
is  a  laborious  process.  The  instructor 
amst  use  a  keyboard  and  paging  functions 
for  data  entry. 

The  data  generated  is  not  readily  available 
nor  is  it  in  a  format  easy  to  interpret 
and  apply  to  the  evaluation  of  the 
student's  progress. 

Another  performance  measurement  feature 
in  a  recently  delivered  flight  simulator 
is  an  overview  of  the  cockpit  activity  where 
the  trainee's  20  most  recent  actions  are 
listed  on  a  CRT. 

Interviews  with  and  observations  of 
users  of  this  device  indicate  that  the  time 
event  monitor  is  not  very  effective  for  monitoring 
specific  procedures  in  the  cockpit  because: 

Not  all  of  the  20  most  recent  actions 
may  be  applicable  to  the  task  the  instructor 
is  monitoring  and  the  page  tends  to 
become  cluttered  with  inappropriate 
information. 

Certain  procedures  applicable  to  the 
task  may  not  be  represented  on  this 
page  and  must  be  followed  on  another 
area  of  the  Instructor/Operator  Station 
(IOS). 

When  the  student  becomes  very  busy 
in  the  cockpit,  these  action  messages 
appear  and  disappear  more  quickly  than 
they  can  be  read. 

ISS  APPROACH 

Rational  techniques  are  needed  to  detect 
and  assign  meaning  to  the  procedural  events 
and  performance  measurements  in  a  trainer. 
This  information  must  be  made  readily  available 
and  must  be  presented  in  an  easy  to  understand 
format  to  the  user.  While  this  may  not  seem 
to  be  a  difficult  task  on  the  surface  it 
has  not  been  built  into  any  trainer  successfully. 


The  remainder  of  this  paper  describes 
efforts  by  Logicon  personnel  to  incorporate 
a  comprehensive  procedures  monitoring  and 
scoring  facility  in  the  Instructor  Support 
System  (ISS).  The  ISS  is  the  product  of 
a  general  effort  to  improve  IOSs  sponsored 
by  the  U.S.  Naval  Training  Equipment  Center. 

Problem  Management 

In  approaching  this  effort  one  of  the 
first  technical  issues  that  arose  was  the 
sheer  size  of  the  problem  and  computer  processing 
power  required  to  handle  it.  The  F-14  OFT 
has  approximately  350  switches  and  other 
pilot  adjustable  mechanisms  in  the  cockpit. 
If  one  attempted  to  monitor  all  of  them  all 
the  time  and  condense,  evaluate,  and  present 
derived  information  to  the  instructor,  one 
would  require  more  processing  power  than 
is  available  with  any  computer  and  more  software 
than  a  small  army  of  talented  programmers 
could  produce. 

The  only  reasonable  solution  appeared 
to  be  the  creation  of  a  mechanism  whereby 
only  the  subset  of  activity  in  the  cockpit 
that  was  appropriate  at  the  moment  would 
be  monitored  and  processed.  This  resulted 
in  the  establishment  of  "task  modules".* 

These  are  operationally  logical  training 
segments  such  as  takeoff,  departure,  or  hydraulic 
malfunction.  A  training  session/acenario 
is  composed  of  a  group  of  these  task  modules. 
fart  of  each  task  module  are  detectable  "events". 
The  ISS  can  follow  the  scenario  by  watching 
for  these  events.  For  example,  a  combination 
of  events  such  as  gear  transition  to  up, 
flaps  transition  to  up,  speed  increasing 
through  200  knots,  heading  increasing  and 
passing  275  degrees,  "starts"  a  task  module 
representing  a  particular  departure  from 
NAS  Miramar.  When  this  task  module  is  started, 
only  those  flight  parameters  and  procedures 
applicable  to  this  training  task  are  monitored 
and  processed. 

Task  modules  may  also  be  "forced"  to 
run  by  direct  action  of  the  instructor. 
More  than  one  task  module  can  run  at  the 
same  time,  if  the  activities  associated  with 
them  overlap.  This  quality  was  also  used 
to  monitor  vital  safety  functions  at  all 
times  the  aircraft  was  off  the  ground.  A 
task  module  was  defined  whose  start  condition 
was  "weight  off  wheels"  and  stop  condition 


*A  companion  paper,  also  prepared  for  this 
workshop,  entitled  MODULAR  CONTROL  OF  SIMULATORS 
explains  the  concept  of  task  modules  in  depth. 
Although  this  paper  indicates  how  task  modules 
fit  into  procedures  monitoring  and  scoring, 
the  reader  is  encouraged  to  refer  to  the 
other  paper  for  additional  information. 
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was  "weight  on  wheels".  Among  other  items 
it  monitored  the  accelerometer  and  informed 
the  instructor  of  any  aircraft  overstresses. 
It  was  nicknamed  the  "umbrella"  and  kept 
the  1SS  from  looking  stupid  by  not  informing 
the  instructor  if  some  disaster  occurred 
The  maximum  number  of  concurrently  running 
task  modules  in  the  ISS  is  four.  This  constraint 
was  due  to  processing  power  limitations, 
but  it  did  not  hamper  operations. 

Task  Module  Structure 

In  order  to  understand  in  more  detail 
how  the  ISS  monitors  procedures  and  performance 
measurement,  one  must  examine  the  ISS  software 
and  task  module  structure.  The  system  is 
driven  by  a  data  base  consisting  of  task 
BKtdule  data  files.  Each  task  module  is  defined 
by  a  set  of  data  files  which  are  unique  to 
that  individual  task. 

One  of  the  files  within  a  task  module 
set  defines  detectable  events  (or  actions) 
that  are  directly,  functionally,  and  physically 
related  to  the  training  task.  Directly  related 
actions  are  those  that  are  depicted  in  the 
NATOPS  manual  as  part  of  the  procedure  (e.g. 
switch  A  -  off,  switch  B  -  stby,  lever  C 
-  up,  etc.).  Functionally  related  actions 
are  any  actions  which  are  similar  in  function 
but  are  not  part  of  the  procedure,  (e.g. 
in  a  particular  hydraulic  failure  a  secondary 
hydraulic  isolate  switch  is  part  of  the  hydraulic 
system  of  the  aircraft,  but  does  not  play 
a  part  in  the  particular  procedure.)  By 
identifying  this  switch,  a  diagnostic  message 
can  be  created  informing  the  instructor  if 
this  switch  is  mistakenly  thrown.  Physically 
related  actions  are  those  linked  to  switches 
that  are  in  close  proximity  to  the  appro¬ 
priate  switch  and  may  look  the  same  (e.g.  if 
the  wrong  circuit  breaker  was  pulled  in  a 
procedure,  this  identification  would  produce 
a  diagnostic  message). 

In  the  definition  of  these  events  the 
following  considerations  are  made: 

Relationship  of  the  cockpit  device 
value  to  a  reference  value  (greater 
than,  less  than,  etc.). 

Stable  time  (to  avoid  triggering  on 
transient  changes). 

Whether  the  event  is  external,  something 
happening  in  the  trainer,  or  internal, 
a  combination  of  external  events. 

Whether  the  event  is  a  verification 
of  a  value  or  a  change  in  its  state. 

Whether  the  event  has  the  "dropout" 
property  (whether  it  is  considered 
to  be  another  occurrence  when  the  reverse 
of  the  initial  happens). 


Procedures  Monitoring 

A  program  dedicated  to  monitoring  these 
events  is  notified  when  a  task  module  becomes 
active  and  opens  a  file  defining  the  appropriate 
events  for  that  task  module.  In  this  way, 
only  a  select  group  of  switches  are  monitored 
at  any  one  time. 

Whenever  any  of  these  events  have  been 
detected,  this  event  monitoring  program  informs 
another  program,  dedicated  to  the  procedures 
monitoring  function.  Like  the  event  monitor, 
this  program  is  also  informed  when  a  particular 
task  module  becomes  active.  It  opens  a  file 
which  has  defined  the  relationship  of  the 
events. 

This  program  evaluates  actions  with 
respect  to  relationships  and  contingencies 
and  causes  a  variety  of  actions  to  be  taken 
by  the  system.  That  is,  it  can  evaluate 
a  sequence  of  events  such  as  the  case  where 
switches  ABC  must  be  thrown  in  sequence, 
and  C  D  E  must  be  thrown  after  the  the  first 
three  but  not  necessarily  in  sequence.  An 
example  of  this  is  the  engagement  of  nose 
wheel  steering  during  an  F-14  takeoff.  The 
steering  is  optional  prior  to  15  knots,  a 
requirement  between  15  and  80  knots,  and 
must  be  disengaged  prior  to  100  knots. 

With  respect  to  contingencies,  not 
only  must  the  specific  sction  be  identified, 
but  also  the  circumstances  under  which  it 
is  to  take  place  must  be  defined.  That  is, 
when  an  event  occurs,  alternate  actions  msy 
be  specified  depending  on  other  circumstances 
or  contingencies.  For  example,  during  takeoff 
if  the  aircraft  heading  drifts  off  the  runwsy 
heading  by  a  specified  amount  while  the  plane 
is  still  on  the  runway,  an  illegal  action 
has  occurred  and  a  diagnostic  message  is  gener¬ 
ated.  However,  heading  changes  sfter  the 
wheels  are  off  the  runway  are  normal  and 
no  action  is  taken  by  the  system. 

Performance  Measurement 

One  of  the  actions  that  may  be  taken 
by  the  system  as  the  result  of  the  occurrence 
of  an  event  is  taking  measurements  of  flight 
parameters.  For  example,  when  the  aircraft 
reaches  a  certain  altitude  the  system  may 
start  measuring  the  pilot's  ability  to  maintain 
that  level  altitude.  Or  it  may  take  a  single 
measurement  at  a  specific  point.  For  example, 
when  the  aircraft  reaches  Ground  Controlled 
Approach  (GCA)  minimums,  it  may  measure  the 
distance  from  the  centerline  of  the  runway. 
A  performance  measurement  program  is  dedicated 
to  taking  measurements  associated  with  a 
task  module  while  it  is  active  and  when  completed, 
calculates  a  score  for  the  task.  It,  like 
the  other  programs ,  is  informed  when  a  particular 
task  module  is  active. 
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The  measurements  are  each  defined  in 
one  of  the  task  module  files.  They  fall 
into  several  general  categories: 

Continuous.  A  measurement  with  predefined 
start  and  stop  conditions.  The  parameter 
is  sampled  1  to  20  times  per  second. 

Monitor.  A  continuous  measurement 
wherein  a  limit  is  defined.  The  parameter 
is  sample  during  the  time  that  it  is 
outside  the  limit. 

Snap  Shot.  A  single  parameter  is  sampled 
once. 

Procedures  Evaluation 

Performance  measurement  for  normal  and  emergency 
procedures,  because  of  their  nature,  is  handled 
quite  differently  than  flight  parameters. 
The  following  factors,  or  subset  of  them, 
are  considered: 

Critical  errors.  Those  actions  which, 
by  their  omission  or  commission,  will 
produce  catastrophe. 

Recognition  reaction  time.  This  measures 
student  response  to  certain  cockpit 
stimuli  in  the  case  of  aircraft  system 
degradation  and/or  failures. 

Total  procedure  time.  This  measures 
the  total  time  required  to  complete 
the  procedure. 

Percent  of  mandatory  actions  taken. 
These  are  actions  that  are  depicted 
in  the  F-14  NATOPS  manual. 

Percent  of  optional  actions.  These 
are  actions  that  are  not  mandatory 
but  demonstrate  that  the  student  is 
well  prepared  and  well  in  command  of 
the  situation. 

Scoring 

The  performance  measurement  program 
is  informed  when  the  task  module  is  completed 
and  calculates  a  score.  It  does  so  by  comparing 
both  the  flight  parameters  measured  and  the 
procedures  evaluated  with  criteria  unique 
to  the  task  module. 

Scoring  of  performance  by  a  computer 
is  a  controversial  issue  and  is  one  of  the 
first  to  be  questioned  by  students  and  instructors 
alike.  Developers  of  the  ISS  have  attempted 
to  defuse  this  problem  by  putting  the  grading 
criteria  under  control  of  the  user's  connunity 
and  by  making  visible  to  the  instructor  (at 
his  request)  the  measurements  and  calculations 
used  to  derive  the  score. 

The  key  to  this  is  a  scoring  template 
in  which  are  placed  all  measurement  definitions 


and  scoring  formulas.  This  template  is  one 
of  the  files  making  up  the  task  module  definition 
and  is  easily  established  or  modified  using 
a  text  editor.  After  a  task  module  has  been 
executed  the  system  displays  the  template 
in  a  format  similar  to  that  in  which  it  was 
created  but  with  the  addition  of  the  actual 
measurements  taken  and  all  the  figures  used 
in  deriving  the  final  score. 

Figure  1  is  a  template  for  a  task  module 
which  defines  the  SAN  PEDRO  DEPARTURE  from 
NAS  Miramar.  The  first  column  under  "MEASURE" 
are  measurements  that  the  student  is  graded 
on  in  performing  this  departure. 

DME  SNAPSHOT  is  the  distance  from  a 
particular  fix  the  student  is  to  fly  over. 
This  is  a  single  measurement. 

RADIAL  DEVIATION  is  the  ability  to 
fly  a  specific  radial.  This  is  a  continuous 
measurement.  RMS  indicates  that  "root  mean 
squared"  is  the  basic  transform  used  in  processing 
the  data. 

AIRSPEED  DEVIATION  is  ability  to  maintain 
a  specific  airspeed.  This  is  a  continuous 
measurement  and  the  RMS  transform  is  used. 

The  second  column  under  "NOMINAL"  is 
the  reference  value  of  the  particular  measure¬ 
ment.  This  can  also  be  used  to  differentiate 
between  measurements  with  the  same  name.  In 
this  example  the  first  radial  measured  is 
the  280  degree  radial  and  the  second  is  the 
3 00  degree  radial.  Likewise  the  1,  2,  3, 
next  to  the  DME  SNAPSHOT  measurements  indicate 
the  first  second  and  third  fixes  in  the  departure. 

The  "4.0,  3.5,  3.0,  2.5"  columns  are  the 
scores  on  the  Navy's  4.0  scale.  The  grade 
for  a  particular  step  in  a  task  module  is 
derived  by  comparing  the  measured  value  with 
the  range  of  values  below  these  columns. 
For  example,  the  first  measurement  listed 
on  figure  1  is  the  distance  at  closest  point 
of  approach  from  the  first  fix  the  student 
was  to  fly  over.  The  actual  value  measured 
was  .79  miles  recorded  under  the  "VALUE" 
column.  By  looking  across  the  template  opposite 
the  first  DME  measurement  we  see  that  the 
measured  value,  .79,  falls  between  0  and 
2.5  which  is  under  the  4.0  column.  Therefore 
the  student  is  assigned  a  4.0  for  that  measure¬ 
ment.  This  is  recorded  under  the  "GRADE" 
column.  The  last  column  "NT"  is  the  weight 
factor  given  to  that  particular  measurement. 
In  this  case  it  was  worth  10  percent  of  the 
total  grade.  After  all  of  the  steps  have 
been  calculated  in  the  method  described  above, 
they  are  summed  to  form  the  final  grade, 
which  in  this  example  is  2.7. 

This  example  does  not  show  any  procedures 
scoring.  If  it  did,  REACTION  TIME,  X  MANDATORY 
STEPS,  or  the  like  would  appear  as  additional 
measurements.  That  is,  procedures  are  scored 
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Figure  1.  Scoring  Template 


in  the  same  manner  as  flight  parameters. 
An  exception  is  the  CRITICAL  ERROR.  If  one 
of  these  occurs,  the  task  module  is  assigned 
the  lowest  possible  grade  regardless  of  other 
measurements . 

Information  Presentation 

Although  it  may  not  seem  to  have  direct 
application  to  procedure  monitoring,  the 
manner  in  which  the  information  is  presented 
to  the  user  is  very  important  to  the  successful 
application  of  these  features.  The  ISS  approach 
again  centers  around  task  modules.  As  part 
of  each  task  module's  definition,  pictures 
are  defined  in  the  task  module  data  files. 
As  task  modules  become  active  and  are  completed, 
appropriate  pictures  are  added  to  and  removed 
from  the  user's  displays.  Task  modules  covering 
procedures  (e.g.  check  lists,  malfunctions) 
are  composed  of  text  identical  to  the  procedures 
as  depicted  in  the  aircraft  NATOPS  manual. 
As  each  step  in  the  procedure  is  accomplished, 
the  action  is  noted  next  to  the  appropriate 
step. 

Figure  2  shows  two  concurrently  running 
task  modules  on  the  upper  half  of  the  display, 
a  normal  checklist  on  the  left  and  a  malfunction 
on  the  right.  Both  are  procedure  oriented 
and  indicate  that  the  ateps  are  in  progress. 
The  section  below,  labeled  "diagnostics", 
containsmesaages  generated  by  the  procedures 
monitor  program.  These  appear  at  the  bottom 
of  that  section  and  move  up  either  when  displaced 
by  a  new  diagnostic  message  or  when  30  seconds 


have  passed.  All  messages  eventually  disappear 
from  the  top  of  that  section.  The  small 
rectangles  represent  menu  selections  that 
can  be  picked  by  the  user  via  a  touch  mechanism 
associated  with  the  display.  This  is  the 
input /control  mechanism  of  the  ISS. 

SUMMARY 

This  paper  has  described  a  software/hardware 
system  that  is  able  to  monitor  procedures, 
and  score  individually,  and  concurrently, 
the  many  training  tasks  in  a  complex  training 
scenario.  This  system  has  approached  the 
problems  from  the  instructor's  viewpoint 
and  is  intended  to  give  him  understandable 
and  appropriate  information  on  an  uncluttered 
instruc tor/operator  station.  Using  this 
he  will  be  able  to  better  evaluate  the  student's 
performance  and  provide  the  instruction  needed 
to  improve  it. 

The  ISS,  of  which  this  procedure  monitoring 
and  scoring  mechanism  is  a  key  part,  has 
been  under  development  for  several  years 
and  is  now  undergoing  operational  evaluation 
at  NAS  Miramar.  The  concepts  developed  in 
this  prototype  promise  improved  training 
in  existing  flight  trainers  and  can  serve 
as  the  basis  for  instructor/operator  stations 
on  new  simulators. 
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Figure  2.  Running  Task  Modules 
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ABSTRACT 

The  conventional  use  of  instructor  pilots  at  flight  simulators  contains  a 
number  of  problems— a  continuous  training  program  Is  required  because  of  Instructor 
rotation  to  the  assignments,  pilots  are  often  not  Interested  in  being  simulator 
Instructors,  and  they  frequently  require  assistance  from  device  operators.  Further¬ 
more,  to  obtain  maximum  utilization  of  a  simulator  a  large  number  of  instructors 
are  required.  The  use  of  student  self-training  programs,  enabling  a  simulator  to 
be  operated  without  an  instructor  when  required,  would  solve  many  of  these 
problems.  This  paper  presents  the  advantages  of  such  a  capability  and  discusses 
the  related  design  considerations.  It  describes  the  required  instructional 
programs  and  offers  several  solutions  to  the  problem  of  locating  simulator  con¬ 
trols  in  the  cockpit.  In  addition,  it  discusses  the  subjects  of  safety  and 
realism  and  concludes  with  recommendations  applicable  to  future  specifications. 


INTRODUCTION 

Conventionally,  flight  simulators  have 
been  equipped  with  more  or  less  elaborate 
instructor  stations,  located  either  on-board 
(in  the  cabin  behind  the  pilot’s  or 
co-pilot's  seat)  or  remotely  (entirely 
removed  from  the  cockpit).  With  the 
on-board  instructor  station  the  instructor 
is  able  directly  to  observe  the  crew 
members  and  evaluate  their  performance; 
with  the  remote  instructor  station  the 
instructor  observes  the  crew  performance 
through  repeater  instruments  and  CRT  displays. 

In  either  case,  the  trend  has  been  to 
strengthen  and  emphasize  the  role  of  the 
instructor.  Instructor  stations  have  been 
designed  to  enable  the  instructor  to  take 
maximum  advantage  of  the  simulator's  in¬ 
structional  capabilities,  to  provide  him 
with  extensive  information  regarding  the 
progress  of  the  student's  training,  and  at 
the  same  time  to  reduce  his  (the  instruc¬ 
tor's)  workload. 

Yet  there  are  many  problems  inherent 
in  this  arrangement.  Instructors  require 
training--the  more  sophisticated  the 
simulator  the  more  extensive  is  this 
requirement.  Then,  after  a  certain  amount 
of  utilization,  most  instructors  rotate  to 
other  assignments  and  never  return  to 
instructing  on  the  simulator.  Or,  if  they 
go  to  sea  duty  or  other  forms  of  lengthy 
TDY  (temporary  duty),  they  need  retraining 
when  they  return. 

Many  pilots  do  not  want  to  be  simulator 
instructors.  If  the  simulator  is  complex, 
pilots  find  that  learning  and  keeping 


current  on  its  operating  procedures  de¬ 
tract  from  their  concentration  on  keeping 
current  on  the  aircraft.  These  procedures 
are  perhaps  not  difficult  to  learn  but 
apparently  are  impossible  to  retain  with¬ 
out  frequent  practice.  The  problem  is 
exacerbated  by  the  fact  that  many  instruc¬ 
tor  stations  are  equipped  with  a  confusing 
array  of  switches  and  other  controls,  often 
poorly  labeled,  and  offer  a  bewildering 
variety  of  instructional  features,  many 
rarely  used.  Furthermore,  most  pilots 
never  accept  having  to  operate  an  alpha¬ 
numeric  keyboard  with  lengthy  formats 
for  data  entry. 

It  is  possible  to  design  instructor 
station  controls,  displays,  and  programs 
so  as  to  reduce  the  demands  on  instructors. 
The  instructor  station  for  Device  2F132, 
the  F/A-18  Operational  Flight  Trainer, 
is  an  example  of  such  an  effort. (!)  However, 
the  ultimate  solution  to  this  requirement 
has  not  yet  been  demonstrated. 

The  use  of  device  operators  is  a  way 
to  solve  these  problems.  Device  operators 
who  do  not  have  other,  conflicting  duties 
can  keep  current  on  all  of  the  instructor 
station  features  and  procedures  and  can 
assist  instructors  in  many  ways,  from 
serving  as  operator  of  the  alphanumeric 
keyboard  to  handling  all  of  the  complex 
instructional  features.  However,  device 
operators  are  usually  not  provided  at 
on-board  instructor  stations,  probably  in 
order  to  minimize  the  number  of  persons 
in  the  cabin  and  to  conserve  on  the  size 
of  the  instructor  station.  Furthermore, 
if  device  operators  are  always  required, 
in  addition  to  instructors,  the  manpower 
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requirements  for  operating  the  simulator 
become  significantly  increased. 

Another  approach  is  to  hire  former 
pilots--retired  military  or  naval  personnel -- 
to  be  simulator  instructors.  Since  they  will 
presumably  have  no  other  function,  they  can 
keep  as  proficient  on  the  simulator  as 
device  operators  can.  However,  there  are 
issues  involved  in  this  approach--the 
instructors’  credibility  with  the  students, 
the  cost  of  the  additional  personnel,  and 
the  availability  of  qualified  individual s-- 
that  may  militate  against  widespread 
adoption. 

Regardless  of  the  source  of  the  instruc¬ 
tors,  the  requirement  to  have  the  simulator 
manned  by  someone  other  than  the  students 
is  a  limitation  on  its  utilization.  A 
modern  simulator  is  capable  of  operating 
at  least  16  hours  per  day.  To  continually 
provide  an  instructor  for  this  level  of 
operation  for  five  days  per  week  requires  a 
force  of  at  least  six  instructors  if  they 
are  not  active  duty  pilots,  or  approximately 
twice  that  number  if  they  are. 

A  different  solution  to  all  of  these 
problems  is  to  develop  a  concept  of  student 
self-training--not  to  eliminate  all  instruc¬ 
tors,  of  course,  but  to  provide  users  with 
the  capability  to  conduct  some  training 
without  the  presence  of  an  instructor 
being  required. 

CONCEPT 

In  a  simulator  with  a  student  self¬ 
training  capability  the  student  should  be 
able  to  conduct  a  complete  training  exercise 
using  controls  entirely  in  the  cockpit. 

To  this  end  he  should  be  able  to  initialize 
at  a  desired  geographical  location,  fly  a 
pre-planned  flight  profile,  introduce  and 
remove  malfunctions,  operate  simulator 
control  functic  s  such  as  freeze  and  crash 
override,  and  operate  instructional 
features  such  as  demonstrations  and  per¬ 
formance  measuring. 

The  exercise  should  be  part  of  an 
overall  training  syllabus  that  would  include 
both  aircraft  missions  and  simulator  exer¬ 
cises.  The  number  of  self-training  exercises 
with  an  Instructor  being  present  should  be 
planned  in  advance  and  scheduled  in  the 
syllabus.  Self-training  should  never  be  an 
"ad  lib"  use  of  the  simulator,  resorted  to 
only  when  an  instructor  is  absent. 

The  optimum  ratio  between  self-training 
and  instructor-monitored  exercises  in  a 
syllabus  would  depend  on  a  variety  of  condi¬ 


tions.  The  availability  of  trained  instruc¬ 
tors  would  be  a  major  factor.  If  the  number 
of  available  instructors  is  small  and  the 
number  of  students  is  large,  self-training 
exercises  would  comprise  a  substantial  part 
of  the  syllabus.  The  qualifications  of 
the  students  would  be  another  factor.  A 
pilot  who  needs  only  refresher  training 
could  use  more  self-training  exercises  than 
a  recent  flying  school  graduate.  In  general, 
if  instructors  are  used  primarily  for 
initial  indoctrination  and  for  checkrides, 
the  ratio  of  sel  f- training  exercises  to 
instructor-monitored  exercises  could  be 
in  the  order  of  four  or  five  to  one. 

DESIGN  CONSIDERATIONS 

Providing  a  student  self-training 
capability  will  have  a  significant  impact 
on  simulator  design--on  the  instructional 
system  software  and,  to  a  lesser  extent, 
on  the  student  station  Hardware  (i.e., 
the  cockpit) . 

Control  s 

Controls  in  the  student  station  will 
have  to  be  considerably  more  elaborate  than 
for  a  conventional  simulator,  which  may 
have  only  a  Freeze  switch  located  very 
inconspicuously.  How  elaborate  will  depend 
on  the  number  of  functions  provided  for  the 
student--to  the  same  degree  that  the  com¬ 
plexity  of  an  instructor  station  depends 
on  the  functions  provided  for  the  instructor. 
If  cost  considerations  dictate  an  austere 
approach,  student  station  controls  can  be 
kept  to  a  minimum--only  sufficient  for 
initialization,  freeze,  motion  control  and 
emergency  off.  Additional  capabilities 
can  be  provided  at  increased  cost,  although 
the  fact  that  these  capabilities  must  be  in 
the  cockpit,  not  at  an  instructor  station, 
will  have  a  distinctly  limiting  effect. 

Student  station  controls  must  have 
certain  required  characteristics.  The 
first  of  these  requirements  is  that  the 
controls--and  the  instructional  features 
that  they  relate  to--must  be  understandable 
and  easily  operated  by  untrained  individuals. 
In  this  case,  "untrained"  means  not  having 
attended  an  operator's  course  and  having 
been  required  to  read  only  minimum  instruc¬ 
tions  on  the  simulator's  operations. 
Simplicity  and  consistency  in  the  design  of 
simulator  controls  will  be  required  to  a 
degree  far  greater  than  heretofore. 

A  second  requirement  is  that  student 
station  simulator  controls  be  easily 
accessible.  Since  the  student  will  be 
devoting  his  full  attention  to  flying  the 
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simulator.  It  Is  Imperative  that  the  distrac¬ 
tion  of  locating  and  operating  non-aircraft 
related  controls  be  kept  to  a  minimum. 

A  third  requirement,  which  Is  not  easily  , 
reconciled  with  the  previous  one,  is  that 
controls  be  located  as  Inconspicuously  as 
possible.  On  the  premise  that  student  accep¬ 
tance  of  a  simulator  depends  on  how  closely 
Its  cockpit,  sound  effects,  and  "feel"  resem¬ 
ble  the  actual  aircraft.  It  Is  apparent  that 
panels,  switches,  and  Indicators  that  are 
foreign  to  the  aircraft  would  be  an  Irritant 
to  the  student.  Obtrusive  simulator  controls 
In  the  cockpit  would  be  as  distracting  as 
those  that  are  hard  to  find. 


Two  simulators  built  by  Sperry  Systems 
Management  -  SECOR  contain  controls  for 
operation  from  the  cockpit,  although  how 
much  they  will  be  used  for  aircrew  self¬ 
training  has  apparently  not  been  determined. 
The  two  approaches  have  both  similarities 
and  differences,  with  respect  to  scope  of 
capabilities  and  concept  of  operation. 

One  design,  the  CH-53  Operational  Flight 
Trainer,  the  first  unit  of  which  is  scheduled 
to  be  delivered  to  the  U.S.  Marine  Corps  in 
late  summer  1982,  contains  a  small  control 
panel  on  each  side  of  the  cockpit,  above  and 
in  front  of  the  two  side  windows  (see 
Figure  1).  The  panels  are  identical,  each 


Figure  1.  CH-53  OFT  Remote  Control  Panels 


having  push-button  switches  for  Initial 
Conditions  Select,  Freeze,  Motion  Ready/On, 
Emergency  Off,  and  Motion  Emergency  Disable. 
Two  thumbwheel  controls  on  each  panel 
enable  the  students  to  select  a  set  of 
initial  conditions  from  twenty  available. 

The  control  panels  are  not  covered  or 
otherwise  concealed  but  are  located  above 
the  pilots  and  copilot's  normal  forward 
field  of  view. 

The  other  approach  is  found  in  the 
HU-25A  and  HH-65A  Flight  Training  Systems 
currently  being  built  for  the  U.S.  Coast 
Guard  for  delivery  in  early  1984.  A  20-key 
keypad,  provided  primarily  for  the  instructor 
station,  which  is  located  in  the  cabin,  is 
installed  with  a  cable  that  enables  it  to 
be  placed  on  the  pedestal  or  the  floor 
between  the  pilot's  and  copilot's  seats. 

The  keys  include  Freeze,  Override  (for 
crash).  Store  (for  flight  conditions).  Reset, 
Enter,  Clear,  Remove  (for  malfunctions), 
ten  digits,  two  punctuation  marks,  and 
backspace  (see  Figure  2). 


Figure  2.  HU-25A/HH-65A  FTS  Keypad 


In  both  the  CH-53  and  the  Coast  Guard 
trainers  the  students  will  be  able  to  use 
the  instructor  station  controls  to  accomplish 
functions  that  are  not  possible  via  the 
cockpit  simulator  controls.  Conducting 
demonstrations  would  be  an  example  of  such 
a  function.  Also,  if  necessary  one  of  the 
students  will  be  able  to  call  up  displays  on 


the  instructor  station  CRTs  and  use  them  for 
self-training  purposes.  For  example,  the 
students  will  be  able  to  inspect  the  air¬ 
craft  track  depicted  on  an  approach  display, 
after  making  an  approach. 

The  self-training  capabilities  of  these 
simulators  are  facilitated  by  the  fact  that 
they  have  on-board  instructor  stations  and 
two-man  crews.  The  copilot  can  readily  go 
to  the  instructor  station  and  use  the  simu¬ 
lator  controls  as  he  desires  without 
interfering  with  what  the  pilot  is  doing. 

If  the  simulator  had  a  single-place  cockpit 
and  a  remote  instructor  station,  the 
student  station  controls  for  self-training 
would  have  to  be  more  self-sufficient. 

Formats 

The  design  of  the  controls  will  be 
influenced  considerably  by  whether  the  data 
entry  formats  are  page-dependent  or  not. 

For  a  "worst  case"  situation,  in  which  the 
student  will  have  no  access  to  the  display 
system,  the  formats  will  have  to  be  non-page 
dependent,  utilizing  input  codes  for  entering 
and  clearing  malfunctions,  modifying  flight 
parameters  such  as  fuel  load  and  armament, 
and  modifying  environmental  parameters  such 
as  ceiling  and  visibility. 

Since  an  alphanumeric  keyboard  in  the 
cockpit  is  out  of  the  question,  all  input 
codes  will  have  to  be  useable  on  a  keypad. 
This  requirement  suggests  that  the  codes 
should  be  numeric,  but  to  help  the  student 
remember  at  least  the  most-used  ones  the 
keys  on  the  keypad  could  be  labeled  both 
alphabetically  and  numerically,  like  a 
telephone,  and  the  codes  could  be  alpha¬ 
betical  . 

If  alphabetical  codes  are  used,  and 
other  alphabetical  inputs  such  as  N,  E,  S, 
and  W  for  latitude  and  longitude  are 
required,  the  student  will  need  a  shift 
key  to  select  one  of  the  three  letters  on  a 
key  and  to  differentiate  between  letters  and 
numerals.  This  approach  tends  to  complicate 
the  operation  of  the  keypad  but  the  alterna¬ 
tive,  the  use  of  all-numeric  codes,  is  less 
desirable. 

There  is  another  alternative  to  the 
use  of  shift  keys,  i.e.  a  two-digit  code, 
used  inthe  FAA's  Voice  Response  System  for 
telephonic  weather  briefings,  that  identi¬ 
fies  each  letter  on  a  key.  For  example,  the 
letter  "A"  is  selected  by  entering  "2,1" 
(refer  q  to  the  2  key  and  the  first  letter 
on  it),  uimilarly,  "B"  is  "2,2".  This 
method,  however,  will  increase  the  time 
spent  in  entering  all  formats,  and  is  con- 
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sidered  to  be  acceptable  only  if  the  input 
codes  can  be  kept  to  one  or  two  letters  in 
length. 

The  number  of  input  codes  required  will 
depend  on  how  much  capability  is  provided  to 
the  student.  It  is  likely,  however,  that 
the  number  will  exceed  what  can  be  remem¬ 
bered  by  the  student,  particularly  if  he 
is  not  using  the  simulator  frequently. 
Therefore,  a  list  of  input  codes  will  be 
needed  to  be  kept  in  the  cockpit.  This  list 
could  be  part  of  an  abbreviated  checklist, 
similar  to  the  Pocket  Checklist  used  in 
aircraft,  and  could  cover  all  of  the  oper¬ 
ating  procedures  that  the  student  would  need 
to  be  reminded  of. 

Functions 


In  a  self-training  mode  the  student 
could  forego  many  functions  that  are 
available  at  the  instructor  station  in  a 
conventional  simulator.  Also,  many  func¬ 
tions  that  are  usually  accomplished  with 
controls  could  be  performed  via  alphanumeri¬ 
cal  entries  to  the  computer.  These 
principles  will  have  to  be  imaginatively 
applied  to  the  design  of  the  student  station 
in  order  to  keep  the  number  of  controls  to  a 
minimum.  A  minimum-sized  control  unit, 
whether  it  is  a  panel  like  the  CH-53  OFT  or 
a  keypad  like  the  Coast  Guard  simulators, 
is  essential  to  meeting  the  three  required 
design  characteristics  discussed  previously. 

The  following  discusses  a  broad  range 
of  functions  typically  exercised  at  an 
instructor  station  and  describes  how  they 
could  be  best  performed  with  controls  in 
the  cockpit. 

Turn-On  Procedures.  Turn-on  procedures 
can  be  considered  to  include  powering-up  the 
system,  initializing  the  computer  and 
peripheral  equipment,  turning  on  the  visual 
system  if  there  is  one,  and  performing  a 
readiness  test.  None  of  these  procedures 
requires  special  consideration  for  the  self¬ 
training  concept;  they  can  all  be  accom¬ 
plished  by  maintenance  personnel,  who 
presumably  will  always  be  present. 

Initial ization.  In  conventional  simu¬ 
lators,  initialization  is  accomplished  in  a 
variety  of  ways:  thumbwheel  controls, 
pushbutton  switches,  and  alphanumeric  key¬ 
board  or  keypad  entries  are  used,  sometimes 
in  combinations,  to  select  the  set  desired 
and  enter  it  into  the  computer.  Keyboard 
entries  and  pushbutton  switches  are  used  to 
display  the  available  sets  on  a  CRT  at  the 
instructor  station.  If  the  instructor 
wishes,  he  can  modify  the  parameters  by 


making  keyboard  entries. 

For  the  self-training  concept,  keypad 
entries  should  be  used  exclusively,  in  order 
to  minimize  the  need  for  hardware  controls. 

In  the  absence  of  a  CRT  in  the  cockpit,  the 
available  sets  can  be  included  in  the  abbre¬ 
viated  checklist  mentioned  previously. 

The  question  arises  whether  the  student 
should  be  allowed  to  make  on-line  modifica¬ 
tions  to  an  initial  conditions  set  before  it 
is  entered,  and  if  he  is  allowed  to  make 
modifications,  how  he  will  be  able  to  record 
them.  The  simplest  approach  is  to  allow  no 
pre-entry  modifications.  In  this  case,  if 
the  student  is  not  satisfied  with  the  initial 
conditions  he  can  make  parameter  changes 
after  the  set  is  entered.  The  student  will 
then  observe  the  results  directly  on  the 
aircraft  instruments. 

Motion.  Because  of  safety  considera¬ 
tions,  the  motion  system  presents  a  special 
problem  for  the  self-training  concept.  With 
a  six-post  system,  the  operator,  before 
activating  the  system,  must  have  a  positive 
indication  that  access  ladders  are  stowed 
and  maintenance  platforms  around  the  cockpit 
are  cleared.  Even  smaller  systems  have 
safety  problems. 

One  approach,  for  the  self-training 
concept,  would  be  to  make  the  maintenance 
personnel  responsible  for  operating  the 
motion  system--they  could  use  an  intercom 
system  to  communicate  with  the  student  when 
necessary.  On  the  other  hand,  it  would  be 
almost  mandatory  to  provide  a  "disable" 
capability  in  the  cockpit.  If  a  switch  is 
provided  for  that  purpose,  it  could  be 
designed  to  indicate  a  "motion  ready"  condi¬ 
tion  and  to  provide  both  an  "enable"  and 
"disable"  capability  for  the  student  (in 
some  types  of  installation,  this  may  not 
be  feasible;  an  additional  switch  may  be 
required).  Additionally,  the  student  could 
be  required  to  obtain  a  verbal  clearance 
with  the  maintenance  personnel ,  over  the 
intercom,  before  activating  the  system. 

Normal  Operating  Functions.  There  are 
a  number  of  routine  operating  capabilities, 
usually  performed  with  push-button  switches 
at  a  conventional  instructor  station,  that 
must  be  provided  in  the  cockpit,  in  a  way 
that  conserves  hardware  controls  as  much  as 
possible.  The  most  basic  of  these  is 
Freeze;  dedicating  a  push-button  switch  for 
this  function  appears  to  be  mandatory. 
Emergency  Off  is  a  similar  requirement, 
although  the  switch  should  be  guarded  in  a 
way  that  guarantees  against  inadvertent  actu¬ 
ation.  Crash  Override  can  be  accomplished 
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with  a  keypad  entry,  if  it  is  assumed  that 
the  student  will  not  need  to  activate  it  as 
an  immediate  response  to  difficulty.  Slew¬ 
ing,  which  in  a  conventional  simulator 
usually  requires  a  joystick  control  and  a 
pushbutton  switch,  can  be  omitted  in  the 
self-training  mode.  If  the  student  wants 
to  change  the  geographical  position  of  the 
simulated  aircraft  he  can  enter  a  new 
latitude  and  longitude  with  a  keypad  entry 
(or,  with  a  more  sophisticated  program,  a 
radial  and  DME  from  a  NAVAIO).  Or  he  can 
reinitialize.  Although  possibly  not  essen¬ 
tial,  Store  IC's  and  Reset  are  very  useful 
functions;  they  enable  the  instructor  to 
store  the  existing  flight  conditions  at  any 
time  and  then  to  reset  the  simulated  air¬ 
craft  there  when  desired  later.  If  these 
functions  are  provided  for  the  self-training 
concept,  two  pushbutton  switches  would  be 
needed.  Finally,  an  intercom  capability 
with  maintenance  personnel  must  be  provided. 

A  switch  is  not  needed,  however,  if  the 
student's  microphone  is  continually  "hot". 

Instructional  Features.  The  major  in¬ 
structional  features  found  in  most  current 
simulators  are  demonstrations,  checkrides 
(also  known  as  programmed  missions), 
playback,  parameter  record,  and  event  print. 
Probably  the  one  most  applicable  to  the 
self-training  concept  is  demonstrations;  the 
possibilities  for  learning  by  observation 
and  emulation,  through  this  method,  are 
almost  limitless.  All  of  the  actions 
required  to  select,  activate,  and  terminate 
a  demonstration  can  be  taken  with  keypad 
entries  (plus  use  of  the  Freeze  switch  to 
start  after  the  pre-demonstration  initial 
conditions  have  been  attained). 

Checkrides  are  also  very  applicable  to 
the  self-training  concept,  although  they 
will  pose  a  problem  with  respect  to  the 
monitoring  function.  In  a  conventional 
simulator,  the  instructor  uses  the  CRT 
displays  to  monitor  the  progress  of  the 
mission;  if  the  student  makes  a  gross  error 
sufficient  to  confuse  the  computer  and  induce 
it  to  score  a  leg  that  is  different  from  the 
one  that  the  student  is  flying,  the  instruc¬ 
tor  can  correct  the  situation  with  a  Manual 
Advance  switch.  In  the  self-training  mode 
it  is  not  feasible  to  provide  the  student 
with  this  capability  or  to  expect  him  to 
monitor  the  mission  leg  by  leg.  It  is 
essential  that  the  missions  be  very  care¬ 
fully  designed  so  as  to  reduce  to  a  minimum 
the  need  for  instructor  intervention;  if 
the  mission  cannot  be  completed  as  intended. 
It  will  have  to  be  aborted. 

Playback  (Dynamic  Replay)  will  be  of 
lesser  usefulness  without  an  instructor 


present  to  contribute  comments.  The  student 
will  know  generally  what  his  errors  were; 
redoing  the  maneuver  or  procedure  correctly 
will  be  his  primary  concern.  If  desired, 
however.  Playback  can  be  operated  with  keypad 
entries,  assisted  by  the  Freeze  switch. 

A  variation  of  Playback  called  Minute 
Replay,  or  sometimes  Instant  Replay,  enables 
the  instructor  to  store  a  number  of  segments 
of  flight  history,  usually  of  one-minute 
duration  each,  and  to  replay  them  during  the 
critique  period  after  the  mission.  This 
capability  is  not  considered  to  be  important 
in  the  self-training  mode.  To  provide  it, 
however,  a  switch  will  be  needed  to  enable 
the  student  to  quickly  select  the  segment 
to  be  stored. 

Parameter  Record  is  a  little-used 
data-gathering  capability.  To  operate 
Parameter  Record  the  instructor  must  select 
individual  parameters  to  be  monitored, 
specify  the  reference  value  and  tolerances, 
and  start  and  stop  the  recording.  These  are 
time-consuming  functions  that  are  not  at  all 
suitable  for  a  student  to  perform. 

Similarly,  Event  Print  is  not  appro¬ 
priate  for  the  sel  f-tra ining  concept.  This 
feature  prints,  either  on  the  Parameter 
Record  printout  or  independently,  the  time 
o*  occurrence  of  a  number  of  events  that  the 
instructor  selects  in  advance.  To  be  useful 
this  capability  must  be  part  of  a  very 
detailed  analysis  and  critique  of  the  mission 
which  only  an  instructor  can  conduct. 

Mai  function  Control  .  As  in  conventional 
simulators,  malfunctions  can  be  entered 
and  cleared  in  the  self-training  mode  by 
using  keypad  entries.  The  list  of  program¬ 
mable  malfunctions  can  total  several  hundred 
items  depending  on  the  type  of  aircraft  and 
amount  of  attention  to  this  subject  desired 
by  the  user.  A  shorter  list  can  be  devised 
for  the  self-training  mode  and  included  in 
the  student's  abbreviated  checklist.  A 
Malfunction  Override  control,  if  desired, 
will  require  a  pushbutton  switch. 

Critics  of  self-training  could  say  that 
considerable  training  value  is  lost,  due  to 
the  absence  of  surprise,  if  the  student 
enters  the  malfunction  himself.  However, 
al 1  training  value  is  not  lost,  particularly 
with  respect  to  practicing  the  steps  in 
emergency  procedures.  Furthermore,  it 
would  be  possible  to  enable  the  student  to 
program  a  series  of  malfunctions  to  occur 
randomly  during  a  mission,  thereby  assuring 
that  he  would  at  least  not  have  foreknowledge 
of  when  a  malfunction  would  occur.  Also, 
programmed  malfunction  “packages",  with 


different  types  of  malfunctions  and  levels 
of  difficulty  depending  on  the  mission  and 
student  qualifications,  could  be  entered 
by  the  maintenance  personnel. 

Miscellaneous.  Additional  functions 
found  in  conventional  simulators  can  be 
accommodated  in  the  sel f- training  mode  with 
keypad  entries  or  omitted  entirely  in  the 
interest  of  simplicity.  A  volume  control 
for  sound  effects,  for  example,  can  be 
omitted  (if  considered  necessary  this 
function  can  be  accomplished  with  a  keypad 
entry).  Controls  for  the  voice  recorder, 
if  one  is  provided,  can  be  omitted — 
separate  control  is  not  needed  in  the  self¬ 
training  mode.  The  catapult-hold  and  -fire 
functions,  for  a  carrier-based  aircraft, 
can  be  accomplished  with  keypad  entries. 
Alternatively,  catapult  fire  can  be  program¬ 
med  to  occur  a  few  seconds  after  the  pilot 
gives  the  "ready"  signal  by  turning  on  his 
exterior  lights,  or  In  another  way,  after 
he  lowers  the  hand-grip.  Similarly,  chocks, 
external  power,  starting  air,  rough  air,  and 
arresting  gear  can  be  provided  via  keypad 
entries.  A  warning  horn  for  a  malfunction 
of  the  simulated  oxygen  system,  provided 
at  the  instructor  station  in  conventional 
simulators  for  safety  reasons,  can  also  be 
installed  at  the  student  station  (if,  in 
fact,  it  is  truly  needed). 

Optimum  Design 

In  summary,  it  is  concluded  that  the 
most  desirable  functions  for  the  self¬ 
training  concept  could  be  accomplished  with 
a  keypad  containing  20  keys.  The  following 
functions  are  considered  to  be  the  most 
useful:  Motion  Ready,  Motion  On,  Emergency 
f  f,  Freeze,  Malfunction  Override,  Store  ICs, 
Reset,  ten  digits.  Enter,  Clear,  punctuation, 
and  backspace. 

The  20  keys  could  be  accommodated  in  a 
unit  similar  to  the  hand-held  terminal  manu¬ 
factured  by  the  Termiflex  Corporation. 

Various  models  of  the  Termiflex  have  the 
capability  of  displaying  one  or  two  lines  of 
alphanumeric  text,  each  10  to  12  characters 
in  length.  This  feature  would  be  essential 
for  the  sel  f- training  concept,  to  enable  the 
student  to  verify  his  input  codes  and  values 
before  entering  them  into  the  computer. 

Three  shift  keys  would  provide  access  to 
the  letters  and  punctuation.  An  example  of 
a  self-training  terminal  is  shown  in 
Figure  3. 

The  terminal  should  have  some  features 
not  found  in  the  current  models  of  the 
Termiflex.  The  keys  should  be  backlighted, 
to  enable  reading  when  the  cockpit  is 


darkened  for  simulated  night  operation. 
Further,  certain  function  keys  (Freeze, 

Motion  Ready/On,  and  Malfunction  Override) 
should  be  brightened  when  selected,  to 
provide  a  status  indication  to  the  student. 
Also,  the  Emergency  Off  key  should  be 
guarded,  preferably  with  a  cover  that  would 
have  to  be  lifted  before  the  key  is  actuated. 

A  hand-held  terminal  eliminates  the 
problem  of  finding  a  blank  panel  in  the 
cockpit  behind  which  to  conceal  the  switches. 
The  terminal  could  be  hung  behind  the  seat 
or  stowed  under  it,  seat  design  permitting, 
and  could  be  removed  entirely  for  instructor- 
monitored  missions  if  it  had  a  plug-in 
capability  similar  to  the  Termiflex. 

Pi  spl  ays 

In  the  foregoing  discussion  of  simulator 
functions  and  the  options  available  for 
accomplishing  them,  it  has  been  assumed  that 
the  CRT  displays  normally  found  at  an 
instructor  station  would  not  be  available 
in  the  cockpit.  However,  it  is  possible 
to  display  both  graphic  and  alphanumeric 
data  to  the  student  via  the  visual  system. 
This  approach  has  been  used  modestly  in  the 
A-6E  Night  Carrier  Landing  Trainer  (NCLT), 
Device  2 FI 22 ;  one  display,  depicting  a  plot 
of  certain  parameters  (vertical  velocity, 
bank,  pitch,  rpm,  glide  slope,  and  center- 
line)  recorded  during  an  approach,  can  be 
shown  on  the  two  visual  display  CRT's  in 
front  of  the  pilot  and  B/N. 

In  the  NCLT  the  alphanumeric/graphic 
display  replaces  the  visual  scene;  an 
alternative  method  is  used  in  the  A-4M  OFT, 
Device  2F108/2B34F--a  line  of  weapon  scoring 
data  is  superimposed  across  the  bottom  of 
the  visual  scene.  The  parameters  include  a 
hit  or  miss  evaluation,  bearing  and  range  of 
the  input,  airspeed  at  release,  and  others. 
This  information  is  available  only  on  the 
instructor  station  visual  monitor,  but  it 
could  be  easily  displayed  to  the  pilot  if 
desired . 

The  possibility  of  using  the  visual 
system  for  alphanumeric/graphic  displays 
provides  almost  unlimited  opportunities 
for  enhancing  self-training.  If  a  side¬ 
viewing  visual  CRT  is  provided  in  the 
cockpit,  it  could  be  used,  as  frequently 
as  desired  by  the  student,  for  any  of  the 
displays  available  at  the  instructor  station. 
A  front-viewing  visual  CRT  could  also  be 
used,  but  in  a  more  restricted  way. 

Of  predictable  interest  to  the  student 
would  be  the  procedure  monitoring  displays 
of  normal  and  emergency  procedures  showing 
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the  results  of  the  student's  attempts,  the 
approach  displays  showing  the  various  pub¬ 
lished  approaches  and  a  record  of  the 
aircraft  track,  and  GCA/CCA/ILS  displays  with 
both  horizontal  and  vertical  projections 
of  the  aircraft  track.  Static  data  such  as 
the  list  of  Input  codes  and  the  contents 
of  the  initial  conditions  sets,  which  would 
be  available  as  instructor  station  displays, 
could  also  be  presented  in  the  cockpit, 
rather  than  in  a  Pocket  Checklist  as 
suggested  previously.  Special  displays 
providing  checklist  instructions  for  oper¬ 
ating  the  simulator  could  be  included. 

If  displays  are  available  to  the 
student,  it  is  possible  that  he  will  want  to 
print  the  most  significant  ones  for  post¬ 
mission  analysis.  Checkride  summary  data 
would  be  particularly  useful.  In  any  event, 
the  display  printout  function  could  be 
operated  by  a  keypad  entry. 

Safety 

In  any  discussion  of  students  operating 
a  simulator  without  an  instructor,  concern 
is  usually  expressed  regarding  safety. 

There  are  a  number  of  potential  sources  of 
danger  around  a  simulator--high  voltage 
electricity,  high  pressure  hydraulics, 
the  motion  system,  and  possibly  others. 
However,  most  of  these  should  be  of  concern 
only  to  maintenance  personnel.  There  should 
be  no  reason  for  students  to  be  exposed  to 
high  voltage,  for  example.  In  any  event, 
maintenance  personnel  will  always  be  present 
during  self-training  periods  and  can  be  made 
responsible  for  enforcing  restrictions  on 
access  to  hazardous  areas. 

The  only  source  of  danger  of  importance 
to  the  self-training  concept  is  the  motion 
system.  The  danger,  of  course,  is  to 
personnel  working  outside  and  in  the 
vicinity  of  the  student  station.  As  dis¬ 
cussed  previously,  the  best  solution  to  this 
problem  is  to  have  motion  system  turn-on 
procedures  involving  both  the  students  and 
the  maintenance  personnel.  Communication 
between  the  two  is  a  necessary  element  of 
these  procedures . 

CONCLUSIONS 

Excluding  the  basic  flight  training 
period,  most  training  in  flying  is  conducted 
without  the  immediate  presence  of  an 
Instructor.  This  statement  is  made  on  the 
premise  that  flying  to  maintain  proficiency 
is  essentially  training.  Throughout  his 
career  a  pilot  receives  instrument  checks, 
annual  proficiency  checks,  upgrade  training, 
and  refresher  training,  during  all  of  which 


an  instructor  or  supervisor  is  very  much  in 
evidence;  but  in  most  of  his  flying  he  is 
expected  to  train  himself--to  assimilate 
guidance  and  instructions  on  the  ground  and 
to  apply  them  in  the  aircraft.  Especially 
in  practicing  takeoffs  and  landings,  in 
instrument  flying,  and  in  weapon  delivery, 
he  learns  primarily  by  self-correction  and 
repetition.  In  brief,  he  is  training  by 
acquiring  experience. 

It  follows  that  a  pilot  can  learn  in 
a  simulator  also  by  these  methods.  What 
makes  self-training  effective  in  an  aircraft 
is  the  pilot's  conscientiousness  and  pro¬ 
fessionalism;  the  same  characteristics  in  the 
student  can  make  self-training  effective  in  a 
simulator.  There  is  »  difference  in  the  two 
situations,  of  course.  The  penalty  for  lack 
of  motivation  or  a  lapse  in  concentration  is 
sometimes  fatally  severe  in  an  aircraft;  in 
a  simulator  there  is  no  penalty  other  than 
wasted  time  and  a  relatively  small  amount 
of  money. 

A  controversial  aspect  of  self-training 
is  the  effect  of  the  loss  of  realism  caused 
by  the  student's  frequent  interacting  with 
the  simulator.  From  time  to  time  during  a 
self-training  exercise,  the  student  will 
take  such  actions  as  reinitializing  the 
simulator,  entering  and  clearing  malfunc¬ 
tions,  storing  flight  conditions  and 
resetting  to  the  stored  conditions,  and 
adding  fuel  and  armament.  In  addition  he 
may  display  emergency  procedures  via  the  CRT 
system  and  review  them  before  entering  mal¬ 
functions,  freeze  the  trainer  and  study  the 
pertinent  approach  plates,  and  interrupt 
the  flight  and  call  up  a  demonstration.  All 
of  these  actions  would  be  foreign  to  his 
procedures  as  a  pilot.  The  question  is, 
would  they  reduce  the  transfer  of  training 
expected  from  the  simulated  control  re¬ 
sponses,  instrument  readings,  and  visual 
scenes  that  he  is  also  perceiving? 

It  is  believed  that  the  answer  to  that 
question  is  negative.  Again,  professionalism 
and  conscientiousness  will  enable  the  student 
to  bridge  the  gap  between  managing  his  own 
training  and  receiving  effective  training 
experiences.  One  ingredient  is  necessary: 
the  management  task  must  not  become  too 
engrossing. 

A  related  concern  is  whether  the  design 
of  the  student  station--and  the  associated 
software--is  sufficiently  simple.  It  would 
be  possible  to  overwhelm  the  student  by 
attempting  to  provide  all  the  capabilities 
of  an  instructor  station.  Designers  will 
have  to  keep  in  mind  the  three  required 
characteristics  for  student  station  controls 
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cited  previously—simpl icity,  accessibility, 
and  inconspicuousness--and  the  necessity  to 
be  conservative  in  providing  instructional 
features.  The  need  for  disciplined  and 
responsive  human  factors  engineering  will  be 
possibly  greater  than  for  any  other  area  of 
development  of  a  simulator. 

In  conclusion,  it  is  recommended  that 
specifications  for  flight  simulators  (cockpit 
procedures  trainers,  operational  flight 
trainers,  weapon  system  trainers,  team 
tactics  trainers,  and  similar  training 
devices)  include  provisions  for  a  student 
self-training  capability,  in  addition  to  a 
conventional  instructor  station.  The  users 
will  gain  tremendous  flexibility  in  simulator 
utilization.  In  the  final  analysis,  they 
will  be  able  to  schedule  simulators  like 
aircraft— simply,  some  flights  can  be  solo 
and  some  dual . 
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Flight  simulation  has  been  extensively 
used  in  both  military  and  commercial  aviation 
training  for  the  past  several  decades.  Every 
year  the  complexity  and  versatility  of  flight 
simulators  grow.  However,  the  full  training 
potential  of  these  devices  is  not  often 
realized.  There  are,  of  course,  a  variety  of 
reasons  why  this  situation  persists.  Lack  of 
transfer  of  training  due  to  procedural  or 
configurational  similarities  between  the 
simulator  and  parent  aircraft,  lack  of 
adequate  out-the-windcw  visual  displays,  and 
the  completeness  of  the  mission  relevant 
aspects  of  the  simulations  are  but  a  few. 
However,  the  point  of  this  paper  is  that  the 
training  potential  of  flight  simulators  could 
be  enhanced  by  improving  the  knowledge  and 
understanding  of  the  operating  principles  of 
the  devices,  by  understanding  instructional 
arrangements  and  by  utilizing  instructional 
skills. 

Lack  of  realization  of  the  full  training 
potential  of  many  simulators  varies  a  great 
deal  from  one  simulator  to  another.  This 
variance  is  created  by  the  complexity  of  the 
mission  being  simulated,  the  age  and  sophis¬ 
tication  of  the  simulator,  the  instructor/ 
operator  station  design,  and  the  availability 
of  instructional  aids. 

In  this  paper,  a  solution  to  the  problem 
is  explored — the  development  of  generic 
instructor  training  packages  for  flight 
simulators.  In  the  following  sections  of  the 
paper,  the  several  general  topical  areas 
relating  to  generic  instructor  training  are 
presented.  These  include  the  advantages  of 
simulator  training,  the  principles  of 
effective  instruction  related  to  simulator 
instructional  features,  and  possible  imple¬ 
mentation  and  delivery  strategies. 

Advantages  of  Simulator  Training 

Ground-based  flight  simulators  play  an 
ijiportant  role  in  the  overall  training  scheme 
of  most  civilian  and  nearly  all  military 
aviation  training  programs.  The  advantages 
of  simulator  training  are  clear.  It  offers  a 
means  to  provide  cost  effective  and  instruc¬ 
tional  ly  effective  training  in  a  safe  environ¬ 
ment.  Cost  savings  are  realized  in  both  the 
obvious  and  same  not  so  obvious  training 
parameters.  Obviously,  fuel  and  maintenance 
costs  associated  with  actual  flight  are 
conserved.  Not  so  obviously,  tremendous 
savings  in  time  are  realized  in  simulator 
training.  These  savings  are  observed  in  both 
student  and  instructor  time  requirements. 
Students  get  more  relevant  training  per  hour 
of  "flight"  because  they  can  concentrate  on 


the  goals  of  a  particular  mission  while 
instructors  may  supervise  much  more  relevant 
training  per  hour  of  their  commitment. 
Simulators  also  allow  a  safe  training 
environment.  In  the  area  of  safety,  the 
advantages  are  clear  and  obvious — simulators 
allow  practice  in  flight  activities  where 
the  only  negative  consequences  are  learning 
with  some  embarrassment  and  where  realistic 
responses  to  emergency  or  compound  emergency 
situations  may  be  practiced.  Last,  but  from 
the  point  of  view  of  this  paper,  by  no  means 
least,  simulators  provide  a  necessary 
hands-on  instructional  format  for  flight 
operations.  With  simulators  as  a  part  of 
the  overall  flight  training  regimen,  the 
opportunity  is  presented  to  allow  hands-on 
and  associated  cognitive  learning  experi¬ 
ences  of  the  proper  training  density  in 
proximity  to  the  preceding  academic  events 
and  to  the  subsequent  inflight  events. 
These  simulator  events  are  particularly 
valuable  because  they  provide  practice  with 
appropriate  feedback  for  the  learning 
objectives  of  interest,  as  well  as  allcw 
repeated  exposures  to  particularly  difficult 
and  important  segments  of  a  mission. 

Needless  to  say,  a  component  of  any 
instructor  training  course  related  to  flight 
simulation  should  be  a  thorough  discussion 
of  these  and  other  advantages  of  simulation. 

Instructional  Principles  and  Instructional 
Features 

A  generic  simulator  instructor  training 
program  should  airplify  a  few  basic  prin¬ 
ciples  of  effective  instruction  particularly 
as  they  relate  to  the  usual  set  of  instruc¬ 
tional  features  found  on  most  in-service 
flight  simulators.  The  following  is  a  list 
of  such  principles  with  their  relationship 
to  seme  commonly  available  instructional 
features. 

1.  Student  Preparation  Leads  to 
Effective  Learning.  It  is  a  well  estab- 
lished,  even  intuitively  obvious,  fact  that 
the  better  prepared  the  student  is  when  he 
cares  to  a  simulator  exercise,  the  more  he 
will  profit  fran  it.  A  generic  simulator 
instructor  training  program  should  be 
designed  to  emphasize  this  point  as  well  as 
to  key  the  instructor  to  use  seme  conmonly 
available  student  preparation  aids.  These 
aids  would  include  the  formal  simulator 
exercise  guides  and  prorpts /brief ing  aids. 
Formal  simulator  exercise  guides  are  univer¬ 
sally  available  in  all  Navy  training 
squadrons.  Seme  are  squadron  developed 
while  others  are  the  result  of  contractor 


supported  ISD  development  activities.  The 
generic  training  should  emphasize  that 
through  the  use  of  these  guides  both  the 
instructor  and  the  student  will  be  prepared 
for  the  content  and  flow  of  the  activities 
in  the  exercise.  Also,  depending  on  the 
sophistication  of  the  documents,  the  student 
may  have  an  excellent  feeling  for  the  level 
of  proficiency  he  is  expected  to  demon¬ 
strate.  At  least  one  simulator  program,  the 
P-14  OPT  program  at  NAS  Miramar,  has  been 
supported  by  a  very  sophisticated  device,  the 
Instructor  Support  System,  which  allows  CRT 
displayed  briefing  adds.  Aids  of  this  type 
could  become  part  of  a  device-resident 
generic  instructor  course  or  any  course  in 
support  of  a  new  IQS  design  that  is  suffi¬ 
ciently  complex  to  contain  it. 

2.  Accurate  and  Timely  Feedback  Leads  to 
Improvements  in  Performance.  Another  well 
known  principle  of  effective  instruction  is 
that  accurate  and  timely  feedback  leads  to 
improvements  in  performance.  A  properly 
designed  simulator  instructor  training  course 
would  emphasize  this  point  and  relate  it  to 
the  several  instructional  features  which  are 
ocrnnonly  available  on  most  simulators  that 
allow  accurate  and  timely  feedback.  These 
instructional  features  include  freeze  record/ 
replay,  automated  performance  measures, 
graphics  displays  and  hardcopy  printouts. 
The  freeze  function  allows  immediate  feedback 
for  the  development  of  fine  motor  control  as 
well  as  directional  control.  It  is 
particularly  useful  because  it  can  be 
employed  in  both  real  time  or  during  a 
performance  replay  session.  Record/ replay 
has  an  obvious  application  and,  in  fact, 
provides  the  basis  of  many  of  the  other 
instructional  features.  Automated  perform¬ 
ance  measures  provide  an  excellent  oppor¬ 
tunity  for  lessening  the  workload  of  the 
instructor  while  providing  a  more  accurate 
and  oanplete  record  of  performance  than  even 
the  most  diligent  instructor  is  capable  of 
providing.  However,  while  the  value  of 
automated  performance  measures  should  be 
emphasized  in  a  generic  simulator  instructor 
training  program,  it  must  be  noted  that  the 
value  of  this  instructional  feature  depends 
on  the  amount  of  instructor  inputs  to  the 
variables,  the  weights  of  the  measures  and 
the  accuracy  of  the  computations.  While 
automated  performance  measurement  systems  are 
not  generally  available  for  most  current 
simulators,  it  is  clear  that  they  will  be 
available  in  future  systems  or  in  outboard 
support  systems.  Graphics  displays,  parti¬ 
cularly  those  which  display  approach,  weapons 
delivery,  and  pattern  information,  are  an 
excellent  means  to  provide  post-session 
performance  feedback.  These  displays  are 
fairly  canton  on  many  current  simulators, 
although  some  are  fairly  unsophisticated . 
However,  future  simulator  instructor  stations 
and  support  systems  for  current  simulators 


will  include  these  displays.  A  clear  em¬ 
phasis  should  be  placed  on  the  value  and  use 
of  these  displays  in  a  generic  instructor 
course.  Hardcopy  performance  printouts  are 
almost  universally  available  cm  all  complex 
simulators.  However,  they  are  almost 
universally  unused  as  an  instructional 
feature.  A  generic  instructor  training 
course  would  emphasize  the  potential  value 
of  these  printouts  as  well  as  provide 
suggestions  about  hew  they  could  be  sum¬ 
marized  to  be  a  more  useful  resource.  Here 
again,  the  value  and  acceptance  of  these 
printouts  depends  on  the  legitimacy  of  the 
measures  expressed. 

3.  Practice  Improves  Performance. 
Another  well  established  principle  of 
effective  instruction  is  that  practice, 
particularly  practice  combined  with  accurate 
and  timely  feedback,  improves  performance. 
As  a  matter  of  fact,  it  has  been  conclu¬ 
sively  demonstrated  that  practice  beyond  the 
point  of  proficiency  is  effective  in  consol¬ 
idating  learning  and  making  it  less  likely 
that  deterioration  in  future  performance 
will  occur.  Opportunities  for  practice  are 
provided  by  both  supporting  materials  and 
instructional  features.  That  is,  within  the 
supporting  materials  designed  for  specific 
training  goals,  the  training  objectives  have 
been  arranged  to  provide  practice  that  is 
related  to  the  difficulty,  criticality,  and 
frequency  of  the  psychcmotor  response  in 
question.  Fran  the  point  of  view  of  in¬ 
structional  features,  the  reset  function  is 
a  key  feature  that  allows  repeated  practice 
for  difficult  or  problem  maneuvers.  Reset 
is  a  commonly  available  feature  which  allows 
the  instructor  to  reinitiate  a  training 
routine  repeatedly  with  constant  or  variable 
flight  conditions  until  satisfactory  per¬ 
formance  is  observed.  Malfunction  insertion 
is  another  practice-relevant  instructional 
feature.  With  this  commonly  available 
feature,  the  instructor  can  manually  or 
automatically  insert  aircraft  malfunctions 
at  any  stage  in  a  mission.  These  malfunc¬ 
tions  may  be  presented  once,  or  repeatedly, 
at  the  instructor's  option.  This  feature 
allows  excellent  practice  in  dealing  with 
emergency  situations . 

A  generic  simulator  instructor  training 
program  would  emphasize  the  importance  or 
use  of  these  practice- relevant  features. 
The  course  would  include  clear  direction 
relating  to  the  use  and  adherence  to  the 
supporting  materials.  Several  problem 
analytic  studies  have  shewn  that  departure 
from  the  training  routines  specified  in 
supporting  materials  is  a  problem  in  many 
Navy  training  squadrons.  The  course  would 
also  include  direction  relating  to  the 
proper  use  of  the  reset  and  malfunction 
insertion  features.  This  direction  should 
emphasize  that  resetting  is  an  excellent  and 
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timely  method  to  provide  repeated  practice 
for  problem  maneuvers.  Qiphasis  would  be 
placed  on  the  proper  use  of  the  malfunction 
insertion  features.  Here  the  theme  would 
center  on  adherence  to  required  malfunction 
insertions  specified  on  supporting  materials 
and  the  proper  use  of  random  malfunction 
insertion.  The  generic  course  would  make  it 
clear  that  intentionally  overloading  the 
student  with  compound  emergencies  is  unreal¬ 
istic  and  may  lead  to  confusion  and 
restricted  skill  advancement  in  the  areas  of 
primary  interest. 

4.  Guidance  and  Motivation  Improves 
Performance.  A  final  generally  accepted 
principle  of  effective  instruction  related 
to  simulator  training  is  that  proper  guid¬ 
ance  and  motivation  improves  performance. 
Surely,  one  of  the  most  important  jobs  of 
the  simulator  instructor  is  to  provide 
proper  guidance  and  direction  to  the 
student.  The  generic  course  would  emphasize 
that  this  guidance  must  be  organized, 
mission  relevant,  and  provided  in  an  in¬ 
structional  mode.  That  is,  it  must  be 
provided  in  a  cooperative  setting  charac¬ 
terized  by  joint  achievement  student/ 
instructor  goals.  Several  instructional 
features  relate  to  the  organization  of 
student  guidance  in  the  simulator  setting. 
These  include  use  of  the  trainer  supporting 
materials  and  use  of  the  prompts /brief ing 
aids.  These  features  provide  a  vehicle  for 
guidance.  However,  the  key  vehicle  for 
guidance  is  a  sensitive  student/ instructor 
relationship.  A  generic  course  should 
enphasize  that  point.  Instructor  provided 
motivation  is  another  key  feature  in  the 
student/ instructor  relationship.  A  generic 
course  should  enphasize  the  need  for 
motivational  carmen ts,  rewards,  and  open 
communication.  In  addition,  the  course 
should  emphasize  the  use  of  some  commonly 
available  instructional  features  which  may 
aid  student  motivation.  These  would  include 
the  demonstration  feature,  the  freeze 
feature  and  all  the  features  that  provide 
feedback  about  performance.  A  generic 
course  should  enphasize  that  demonstrations 
can  be  motivationally  effective  if  they  are 
not  overused  or  if  they  are  used  as  a 
feedback  mechanism  to  provide  direction  on 
the  proper  procedures.  The  freeze  feature 
can  also  have  a  profound  motivational  effect 
when  it  is  used  to  intercept  poor 
performance  or  to  indicate  a  "terminal" 
condition  such  as  striking  the  ground  or 
being  hit  by  a  weapon.  The  course  would 
point  out  clearly  that  the  instructor  is 
responsible  for  providing  motivation  through 
the  thoughtful  use  of  all  of  the  performance 
feedback  features.  These  would  include 
automated  performance  measures,  record/ 
replay,  graphics  displays,  and  hardcopy 
printout.  The  erphasis  here  should  be  on 
the  proper  and  sensitive  use  of  these  data 
sources  to  improve  performance. 


Implementation  of  Generic  Training 

There  are  several  potential  advantages 
for  a  generic  simulator  training  program. 
The  program  would  undoubtedly  improve  the 
effectiveness  of  the  instructor  cadre  for 
every  simulator  where  it  might  be  imple¬ 
mented.  This  improved  effectiveness  should 
lead  to  a  more  professional  approach  to  the 
overall  use  of  simulator  training,  or  a  part 
of  flight  training  in  general,  and  to  poten¬ 
tial  improvement  in  student  performance.  In 
addition,  the  course  should  lead  to  improve¬ 
ments  in  student  management  from  both  the 
interpersonal  and  administrative  points-of- 
view.  Interpersonal  improvements  could 
result  from  the  course  content  in  the  areas 
of  guidance,  motivation,  and  performance 
feedback.  Administrative  improvements  could 
result  from  the  generally  improved  organi¬ 
zation  of  simulator  training  and  from  the 
use  of  automated  management  aids  that  might 
be  part  of  a  device-resident  training  course. 

A  disadvantage  of  the  proposed  concept 
is  that  each  instructor  training  course 
associated  with  a  particular  simulator  would 
have  to  be  tailored  to  reflect  the  particular 
instructional  features  and  the  nature  of  the 
supporting  materials  of  that  device.  This 
requirement  admittedly  diminishes  the  generic 
value  of  the  concept  but,  hopefully,  only  a 
small  amount  of  tailoring  would  be  required 
to  address  most  simulators.  The  training 
course  could  be  implemented  in  two  distinctly 
different  formats — device-resident  or  device¬ 
independent.  In  the  device-resident  format, 
the  course  could  be  contained  in  the  software 
of  the  simulator  computer  and  presented  on  a 
CRT  associated  with  the  IOS,  or  a  supporting 
system.  With  this  format, the  course  would  be 
continuously  available,  obviously  convenient, 
and  might  even  be  made  contingent  for  in¬ 
structor  utilization  of  the  device.  In  the 
device- independent  format,  the  course  could 
be  presented  in  a  stand  alone  media,  such  as 
a  workbook  or  slide/tape,  and  could  be  pre¬ 
sented  in  an  individualized  study  atmosphere 
in  a  learning  center  or  training  support 
center.  This  format  would  allow  instructor 
access  in  a  self-paced,  individualized 
arrangement  which  is  consistent  with  the 
manning  demands  of  most  training  squadrons. 
In  either  format,  the  ccurse  would  demand  a 
minimum  of  intrusion  into  training  squadron 
activities  during  its  development  while 
providing  a  iraximuti  of  benefit  for  instructor 
training. 
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INTRODUCTION 

The  objective  of  the  research  reported 
here  is  to  evaluate  the  feasibility  and  useful¬ 
ness  of  applying  two  emerging  technologies  to 
naval  training.  These  technologies  are 
artifical  intelligence  and  voice  recognition 
and  synthesis. 

Artifical  intelligence  (AI)  is  a  body  of 
concepts  and  techniques  which  have  been  devel¬ 
oped  to  permit  computers/machines  to  do  some 
of  the  complex,  cognitive  activities  that  have 
been  regarded  traditionally  as  the  unique 
providence  of  human  beings,  in  some  instances 
exceptionally  talented  human  beings.  Rep¬ 
resentative  cognitive  activities  are  problem 
formulation,  searching  for  problem  solutions, 
diagnosis,  and  decisionmaking.  These 
activities  require  representation  of  knowledge 
and  inferential  reasoning  about  that  knowledge. 

If  these  concepts  can  be  combined  with 
voice  recognition  and  synthesis,  then  an  inter¬ 
active,  oral  dialogue  can  take  place;  the 
verbal  exchanges  can  be  flexible,  meaningful, 
and  appropriate  to  the  context  of  the  on-going 
events.  Applied  to  training  and  education, 
computer-aided  instruction  moves  a  major  step 
in  the  direction  of  a  realistic  journeyman- 
apprentice  relationship.  The  journeyman  can 
be  designed  to  be  not  only  a  technical  expert 
but  an  expert  tutor  as  wel 1 . 

The  organization  of  this  paper  consists 
of  four  parts : 

Review  and  Evaluation  of  the  Component 

Technologies  and  Principal  Concepts  in 

Artifical  Intelligence 

Review  of  Naval  Training  Needs  Suitable 

for  a  Prototype  Course  Using  AI  and  Voice 

Recogni ti on/Synthesis 

Recommended  Guidelines  for  Instructor 

Support 

Discussion  of  Implications  and  Issues 

This  research  has  been  funded  through  two 
sources.  One  source  is  an  on-going,  multi¬ 
year  project  under  Honeywel 1 1 s  program  of 
Independent  Research  and  Development.  The 
second  source  is  a  contract  with  the  Naval 
Training  Equipment  Center  (NTEC  81 -C-0093-1 3) 
entitled  "Use  of  Voice  Technology  a  the 
Instructor's  Assistant." 


It  is  assumed  that  the  demographic  pre¬ 
dictions  of  unavailability  of  future  military 
manpower  are  accurate.  Therefore,  it  will  be 
necessary  to  develop  more  efficient  training 
systems  and  instructional  methodology  that  are 
more  effective  in  amount  of  competence  produced 
per  hour  of  instruction.  The  proposed  approach 
is  feasible  and  will  provide  a  training  capa¬ 
bility  to  meet  the  need.  The  cost  versus 
benefit  is  more  difficult  to  assess,  however. 

We  can  assume  that  software  costs  will  continue 
to  accelerate  and  that  all  technology  is  expen¬ 
sive  to  develop.  Intelligent  systems  of  this 
type  are  software  intensive.  Further,  the 
technological  development  needed  is  undeter¬ 
mined  but  certainly  not  trivial. 

Artifical  intelligence  and  voice  tech¬ 
nology  may  no  longer  be  embryonic  but  they  are 
far  from  mature  for  most  applications. 

REVIEW  AND  EVALUATION  OF  THE  COMPONENT 
TKHNMBGIE5  AND"  PRINCIPAL'  CONCEPTS  IN 

Agnnm  TgTmrcgror  ~ - 

The  needs  of  the  Navy  in  instructional 
capability  were  examined  briefly  for  the 
purpose  of  guiding  the  survey  of  technology. 

The  results  of  that  survey  were  that  the  Navy 
needs  instructor  support,  friendly  and  for¬ 
giving  interfaces  between  the  training  devices 
and  both  the  instructor  and  student,  and 
automated  performance  measurement.  The 
functions  and  tasks  for  which  instructor 
support  is  needed  are: 

Subject  Matter  Expertise 
Performance  Evaluation 
Problem  Selection 
Delivering  Feedback 
Progress  Management 
Documentation 
Role  Playing 

The  review  of  AI  technology  was  organized 
into  the  following  topical  categories: 

Knowledge  Representation 

Learning  Theory 

The  Nature  of;  Expertise 

Intelligence  Computer  Aided  Instruction 

Software  Architecture  for  Artifical 

Intel  1 igence 

Natural  Language  Understanding 
Interactive  Speech 

The  major  conclusions  will  be  summarized 
briefly. 
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The  area  of  Knowledge  Representation  is 
known  as  Domain  Knowledge.  There  are  two 
relevant  domains  for  a  training  system:  the 
knowledge  content  of  the  technical  specialty 
being  taught  and  the  instructional  knowledge 
of  methods,  strategies,  and  procedures.  They 
present  no  unresol vable  problems  for  knowledge 
representation.  The  content  of  the  domains 
w’ll  become  manifest  as  the  training  materi¬ 
als  are  developed.  However,  not  all  of  the 
necessary  knowledge  has  been  explicitly 
articulated  or  well-defined. 


An  important  area  of  instructional 
knowledge  is  the  evaluation  of  student  perfor¬ 
mance.  The  level  of  effectiveness  of  a  train¬ 
ing  system  is  dependent  on  the  availability 
of  two  kinds  of  evaluative  data: 

Assessment  of  the  level  of  performance 
in  terms  of  level  of  competence  and  one's 
standing  in  the  subject  matter. 

Diagnosis  of  errors  committed  in  terms  of 
specific  deficiencies  in  knowledge  or 
skill  and  necessary  corrective  action. 

Existing  techniques  of  knowledge  rep¬ 
resentation  are  adequate  for  constructing  an 
intelligent,  interactive  training  device  to 
support  an  instructor.  These  techniques  con¬ 
sist  of  representing  things  and  events  in  tem¬ 
plates,  frames,  and  scripts  and  the  relation¬ 
ships  among  them  in  control  rules  and  produc¬ 
tion  rules.  These  rules  also  govern  additions 
to  knowledge  derived  internally  by  inference. 

The  contribution  of  Learning  Theory  is 
primarily  in  providing  a  structure  and  method¬ 
ology  for  preparing  training  materials.  One's 
first  expectation  might  be  that  learning 
theory  would  provide  models  of  the  learning 
process;  however,  it  performs  that  function 
only  indirectly.  Learning  theory  will  provide 
a  generic,  transportable  framework  and  methods 
for  developing  courseware  and  rules  and  tech¬ 
niques  for  performing  task  analysis.  These 
methods  and  techniques  entail  specification  of 
stimuli,  acceptable  and  unacceptable  responses, 
feedback,  sequencing,  branching,  display  for¬ 
mats  and  so  on;  they  are  in  turn  a  function  of 
the  domain  content,  training  objectives, 
instructional  strategy,  media,  rules  for  per¬ 
formance  assessment,  and  specific  instructional 
techniques.  All  of  this  is  dependent  on  the 
learning  process.  However,  that  knowledge 
must  be  encoded  into  the  rules  for  instructional 
expertise  and  it  will  be  transparent  in  the 
application. 

Our  interest  in  the  nature  of  expertise 
is  how  the  knowledge  of  competent  practi¬ 
tioners  should  be  used  in  the  training 
system.  Examination  of  the  representation  of 
that  knowledge  revealed  that  the  form  of  rep¬ 
resentation  varies  with  domain.  Domains 


differ  in  structure  encompassing  the  gamut 
from  production  rules  to  hierarchical  organ¬ 
ization;  multiple  conceptualizations  for  the 
same  phenomena  are  available  in  some  domains; 
multiple  levels  of  abstraction  are  character¬ 
istic  of  some  domains. 

Problem  solving  methods  also  varied  across 
domains.  Different  methods  are  available  for 
different  tasks.  A  given  problem  solving 
method  is  also  adapted  to  the  requirements  and 
constraints  of  specific  task  conditions. 

Therefore,  it  seems  unlikely  that  exper¬ 
tise  can  be  captured  in  a  few,  simple,  generic 
algorithms  or  procedures  that  are  generaliz- 
able  across  knowledge  domains  or  even  within 
a  domain.  The  interaction  between  the  nature 
of  expertise  and  knowledge  representation  is 
perhaps  the  most  critical  element  in  develop¬ 
ing  an  intelligent  training  system.  Knowledge 
structures  and  control  rules  that  can  represent 
the  complex  contingencies  will  have  to  be 
developed.  The  subject  matter  to  be  learned 
at  novice  levels  will  probably  be  relatively 
simple  and  without  much  of  the  troublesome 
complexity.  This  problem  must  be  faced, 
though,  when  the  training  moves  into  more 
sophisticated  or  intermediate  level  topics. 

One  perspective  on  the  purpose  for  devel¬ 
oping  these  intelligent  systems  is  to 
shorten  the  transition  time  from  novice  to 
expert.  We  want  to  produce  higher  levels 
of  competence  in  shorter  periods  if  instruction 
and  with  lower  support  costs.  Ideally,  this 
transubstantation  will  be  accomplished  on 
trainees  who  are  naive  with  respect  to  the 
technical  area  and  of  no  more  than  average 
intelligence  (general  intellectual  ability). 

An  instructional  system  that  will  accomplish 
this  feat  will  need  a  model  of  the  sequences 
of  developmental  stages  from  novice  to  expert, 
states  within  those  stages,  and  the, pedagogical 
techniques  for  transformation  between  states 
or  stages.  At  the  present  state  of  our  tech¬ 
nology  such  a  model  must  be  derived  within  the 
context  of  a  specific  domain. 

The  Software  Architecture  for  Artifical 
Intelligence  was  viewed  as  consisting  of  two 
parts:  the  hardware  necessary  to  execute  the 
program  and  the  software.  The  hardware  was 
found  to  be  specific  to  the  training  program 
and  thus  no  generalizations  can  be  made.  It 
was  concluded  that  LISP  is  the  software  system 
that  should  be  used  for  development  and 
operation  of  such  training  systems.  Knowledge 
representation  techniques  exist  in  LISP  as 
well  as  an  assortment  of  supporting  tools  and 
facilities.  Further,  there  is  a  history  of 
experience  with  this  language  which  provides 
both  a  body  of  knowledge  and  experience  A1 
practioners  to  draw  upon. 

Interactive  Speech  and  Natural  Language 
Understanding  present  no  obstacle  in  principle 


although  deriving  and  organizing  the  detailed 
dialogue  requirements  can  be  a  challenging 
management  task.  Spoken  messages  can  be 
synthesized  at  the  current  state-of-the-art. 
Generation  of  the  messages  in  the  context  of 
on-going  events  and  conditions  and  in  an 
adequate  approximation  to  real  time  is  uncer¬ 
tain  and  must  be  determined  in  the  specific 
applications. 

Current  and  near  future  voice  recognition 
systems  can  recognize  utterances  of  only  a  few 
words  in  length;  recognition  and  understanding 
of  unconstrained,  connected  speech  is  not 
feasible  and  may  not  be  practical  for  some 
time.  This  character  of  voice  technology 
imposes  a  requirement  that  the  natural  lan¬ 
guage  dialogue  consist  of  short  statements. 
Short  statements  may  be  characteristic  of 
speech  in  task-oriented  situations  and  thus 
this  constraint  may  not  be  a  practical 
limitation. 

REVIEW  OF  NAVAL  TRAINING  NEEDS  SUITABLE  FOR 
7T  PROTOTYPE  COTJRST - 

Four  training  areas  were  reviewed  for 
suitability.  They  are; 

Electronic  Maintenance 

Air  Intercept  Control 

flaval  Flight  Officer  Course 

Anti-Submarine  Warfare  Team  Training 

In  addition,  training  equipment  in  team 
training  and  air  traffic  control  was  analyzed. 
The  team  trainers  analyzed  were  TACOEW, 

14A2  ASW,  and  14A12;  the  air  traffic  control 
trainers  were  PARTS  (McCauley  and  Semple, 

1980)  and  the  AIC  Trainer  (Halley,  King,  and 
Regelson,  1978). 

The  criteria  used  to  evaluate  the  train¬ 
ers  were  cost-benefit,  availability/shortfall 
of  instructors,  appropriateness  of  content 
to  AI  and  voice,  impact  on  readiness,  and 
stability  of  the  subject  matter.  The  cost- 
benefit  criterion  was  broken  down  into  three 
parts.  First,  the  developmental  cost  of  the 
prototype  was  treated  in  terms  of  the 
availability  of  appropriate  information, 
data,  and  on-going  training.  Second,  operat¬ 
ing  cost  was  treated  as  cost  of  instructors 
as  the  major  factor  differentiating  among 
courses.  The  third  part  was  a  payback  com¬ 
ponent  in  terms  of  readiness  of  trained  per¬ 
sonnel  and  reduction  in  personnel  replacement. 

/’ppropriateness  of  the  subject  matter  for 
the  use  of  artifical  intelligence  and  inter¬ 
active  voice  is  an  important  methodological 
consideration.  The  prototype  application  must 
be  able  to  reflect  an  advantage  from  utilizing 
the  new  technologies.  Artifical  intelligence 
will  be  beneficial  in  domains  that  have  a 
significant  cognitive  component;  voice  tech¬ 
nology  is  useful  in  areas  of  high  workload 


which  can  be  reduced  by  using  hearing  and 
speech  as  media  of  interaction  rather  than 
vision  and  manual  responses. 

ASW  team  training  was  chosen  out  of  the 
four  training  areas  as  the  best  candidate  for 
a  prototype  application.  Electronic  main¬ 
tenance  was  rejected  because  it  does  not  have 
a  significant  voice  potential  and  develop¬ 
mental  costs  could  be  high.  Maintenance 
skills  are  poorly  understood  and  significant 
costs  and  delay  could  be  incurred  in  getting 
necessary  information.  Air  intercept  control 
and  the  naval  flight  officers  course  were 
rejected  because  the  subject  matter  is  not 
stable. 

RECOMMENDED  GUIDELINES  FOR  INSTRUCTOR  SUPPORT 

Analysis  of  the  instruction  yielded 
needs  for  the  following  modules  for 
instructor  support: 

Record  Keeping 
Scenario  Set  Up 
Subject  Matter  Expert 
Subject  Matter  Tutor 
Speech  Generation 
Speech  Recognition 
Ability  to  Understand  Errors 
Omniscient  Instructor 

The  major  components  of  an  Automated 
Training  System  are: 

Instructor  Work  Station 
Student  Work  Station 
Domain  Expert 
Instructional  Expert 
Performance  Measurement 

The  Instructor  Work  Station  was  examined 
in  more  detail  to  identify  the  capability  it 
should  have.  The  Instructor  Work  Station 
should  have  the  following  functions: 

Explanation  of  Decisions  and  Actions 
Record  Keeping  and  Scheduling 
Scenario  Design  and  Debug 
Communications  Control 
Problem  Status 

An  important  capability  of  the  system  is 
to  recognize  and  cope  with  errors  committed 
by  the  trainee.  This  capability  entails 
several  requirements  in  knowledge  and  use  of 
the  knowledge  to  make  inferences  about  the 
trainee's  behavior.  The  system  must  know  the 
constraints  reflected  in  the  rules  for  correct 
behavior.  Since  the  application  is  ASW  team 
training,  the  system  must  also  know  how  the 
trainee  deals  with  other  team  members.  Each 
player  must  know  the  roles  of  the  other 
players  at  a  minimal  level  of  their  nominal, 
expected  behaviors  in  relation  to  one's  own 
role. 
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Therefore,  the  system  must  know  not  only 
the  role  structure  of  the  team  but  also  each 
trainee's  beliefs  about  knowledge  and  com¬ 
petence  of  other  players.  It  must  be  able  to 
follow  a  trainee's  reasoning  about  the  knowl¬ 
edge  of  other  players  and  know  whether  he 
attributes  erroneous  or  unexpected  behavior 
from  another  player  to  a  lack  of  knowledge, 
misinformation,  or  faulty,  missing,  or  in¬ 
appropriate  procedures.  The  inferences  the 
trainee  makes  and  how  he  responds  to  the  other 
player's  behavior  are  relevant;  he  may  fill 
in  missing  information,  correct  the  other 
player,  request  verification,  or  adjust  his 
own  behavior  to  compensate  for  the  perceived 
deficiencies  in  the  other  player's  behavior. 

These  capabilities  have  been  clustered 
into  four  sets  which  represent  increasing, 
levels  capability  and  related  to  three  stages 
in  developing  an  intelligent,  automated  in¬ 
structor  or  instructor's  assistant,  based  on 
Crowe  et  al .,  (1981 ).  We  have  modified  the 
model  of  Crowe  et  al .  by  adding  a  fourth 
stage  consisting  of  the  ability  to  provide 
feedback  and  performance  evaluation  to  team 
behaviors.  The  cross-correlation  of  these 
classifications  is  depicted  in  Table  1. 


The  correspondence  between  the  major 
components  of  an  Automated  Training  System 
and  the  Al  technological  areas  on  which  they 
depend  is  summarized  in  Table  2. 

DISCUSSION  OF  IMPLICATIONS  AND  ISSUES 

Our  intent  in  this  section  is  to  be 
explicit  about  some  issues  and  concerns  that 
were  recurrent  during  our  analyses  and  set 
aside  for  the  purpose  of  attaining  the 
immediate  goal  of  developing  some  guidelines 
for  how  to  proceed  in  developing  an  simulated, 
intelligent,  talking  instructor's  assistant. 
They  should  at  least  be  stated  openly  because 
they  may  represent  potential  problems  or 
obstacles  to  developing,  fielding,  and 
effectiveness  of  this  type  of  system.  The 
intrepid  innovator  can  then  beware  end 
take  appropriate  cautionary  or  preemptive 
actions. 

These  issues  and  implications  are: 

Impact  on  the  Instructor 

Reduction  of  Personal  Contact  between 
the  Student  and  Instructor 

The  Achievable  Level  of  Naturalness 

Effect  of  Complexity  on  Friendliness 
and  Support 

Importance  of  Message  Generation  Versus 
Voice  Synthesis  in  the  User-System 
Interface 

Compressibility  of  the  "Novice-To- 
Expert"  Curve 


There  are  several  possible  effects  on  the 
instructor  and  his  role  which  can  have  a  neg¬ 
ative  impact  on  system  effectiveness.  The 
content  of  instructor  training  will  necessarily 
be  changed  and  perhaps  increased  in  length  and 
difficulty.  Our  intuition  tells  us  that  the 
instructor  will  need  a  better  understanding  of 
the  instructional  process  than  he  presently 
has.  At  the  same  time  his  automated  assistant 
will  permit  him  to  handle  more  trainees  and 
attain  higher  levels  of  proficiency;  his 
productivity  should  undergo  a  marked  enhance¬ 
ment.  However,  without  an  instructional ly 
sophisticated  supervisor  the  automated  in¬ 
structor's  assistance  can  run  amok  like  the 
sorcerer's  apprentice. 

The  instructor's  morale,  involvement,  and 
sense  of  using  his  own  talents  will  be  affect¬ 
ed.  These  factors  will  be  enhanced  if  the  in¬ 
structor  perceives  his  automated  assistant  as 
a  tool  which  helps  him  in  doing  his  job,  as 
something  to  which  he  delegates  tasks  and 
which  increases  his  effectiveness  and  permits 
him  to  do  a  better  job.  If  in  contrast  he 
sees  the  “iron  monster"  as  something  which  he 
must  labor  to  support  and  dominates  his 
activities,  then  his  morale,  motivation,  and 
involvement  will  decline. 

Reduction  of  personal  contact  between  the 
instructor  and  trainee  can  compromise  the 
effectiveness  of  social  reinforcers  and 
attenuate  an  important  source  of  feedback  and 
diagnostic  information  which  every  good  in¬ 
structor  learns  to  use.  It  will  also  minimize 
the  opportunity  to  use  a  mentor  relationship 
to  model  behaviors  and  facilitate  the  instruc¬ 
tional  process.  The  voice  of  the  simulated 
assistant  may  take  on  a  personality  and  a 
reality  as  an  individual  and  thus  be  able  to 
satisfy  some  of  these  social  functions. 
Realizing  them  may  be  a  function  of  the 
creativity  in  preparation  of  the  dialogues, 
the  types  interactions  embedded  in  them,  and 
the  naturalness  of  the  interaction  as  per¬ 
ceived  by  the  trainee.  The  personality  of  the 
automated  assistant  may  be  another  domain  of 
expertise  needed  to  develop  and  deploy  these 
systems . 

The  achievable  level  of  naturalness  is 
unknown  at  this  time  as  is  the  required  level 
of  naturalness.  Naturalness  and  friendliness 
are  concepts  that  are  not  well  defined  and 
have  a  large  subjective,  intuitive  component 
as  the  terms  are  used.  Specifications  are 
difficult  to  state  and  satisfy  under  such 
conditions.  These  terms  need  to  be  differ¬ 
entiated  and  made  more  precise. 

The  impact  of  naturalness  and  the  com¬ 
plexity  of  the  interactions  on  system  support 
and  maintenance  is  a  very  serious  issue.  The 
transparency  of  natural,  friendly,  simple 
dialogues  is  achieved  by  putting  the  com¬ 
plexity  and  processing  behind  the  interfacing 


Stages  of  System  Development 


Technology 


Limited  Role  Student 
Tutor  Play  Model 


Expert  System  X 

Explanation 


Simulation  (With  Errors)  X 

Speech  Generation 


Speech  Recognition 


Understanding  Misconceptions 
&  Intent 

Learning  Model  &  Feedback 


Omniscient 

Instructor 


TABLE  1. 


CORRESPONDENCE  BETWEEN  STAGES  OF  DEVELOPMENT  OF  AN 
AUTOMATED  INSTRUCTOR'S  ASSISTANCE  AND  AI  CAPABILITIES 


console  and  into  the  software.  Experience 
tells  us  that  maintenance  and  support  in¬ 
crease  as  software  and  hardware  become  more 
complex;  Murphy's  Law  is  imperative. 

Voice  synthesis  with  some  degree  of 
natural  sounding  speech  is  relatively  easy  to 
achieve  in  the  current  state  of  the  art. 
However,  the  appropriateness  of  the  message 
in  the  context  of  on-going  events  and  the 
speech  community  of  the  listener  are  more 
difficult  to  accomplish.  These  features  are 
the  problem  of  message  generation.  It  is 
dependent  on  a  lot  of  information  processing 
involving  knowledge  representation,  inference, 
and  domain  knowledge.  This  area  is  critical 
to  the  effective  use  of  voice  technology. 

Compressibility  of  the  curve  for  trans¬ 
forming  a  novice  into  some  level  of  expert 
is  an  ultimate  instructional  issue.  There 
are  many  differences  between  novice  and 
expert  in  amount  of  knowledge,  its  organ¬ 
ization,  techniques  for  formulating  and 
solving  problems,  and  the  use  of  heuristics 
derived  from  experience  and  the  accumulated 
lore  of  the  discipline.  A  subset  of  these 
things  must  be  chosen  in  terms  of  their 
utility  on  the  job  for  a  given  level  of 
proficiency,  the  ease  of  learning  them,  and 
the  progression  of  states  of  knowledge  and 
skill  in  the  growth  of  proficiency.  There 
is  a  multitude  of  empirical  questions  in¬ 
herent  in  this  issue;  they  must  oe  unscramb¬ 
led  and  addressed  if  we  are  to  have  an  effect¬ 
ive  technology  for  automation  of  instruct¬ 
ional  systems. 

CONCLUDING  REMARKS 

Application  of  artifical  intelligence 
and  voice  technology  to  training  systems  in 
the  form  of  an  automated  instructor's 
assistant  is  feasible  and  can  provide  sig¬ 
nificant  increases  in  the  cost-effectiveness 
of  training.  Artifical  intelligence  provides 
several  concepts  and  techniques  which  are 
useful  in  structuring  and  implementing  the 
design  and  use  of  such  a  system.  This  paper 
is  a  brief  summary  of  an  on-going  investi¬ 
gation  of  the  feasibility,  utility,  and 
approach  to  developing  an  automated 
instructor's  assistant  for  use  in  training 
Navy  personnel.  ASW  team  training  was  chosen 
as  the  vehicle  for  a  prototype  application 
of  the  automated  instructor's  assistant. 
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Moffett  Field,  CA  94035 

Dr.  J.  Huddleston 
1  Head  of  Personnel  Psychology 
Army  Personnel  Research 
Establishment 
C/o  RAE,  Farnborough 
Hants,  ENGLAND 

1 

Mr.  David  A.  Citrin 

General  Electric  Ordnance  Systems 

Rm  2372 

3  100  Plastics  Avenue 

Pittsfield,  MA  01201 

Mr.  Ron  Handy 
Pacer  Systems 
1  87  2nd  Avenue 

Burlington,  MA  01803 

Corporate-Tech  Planning  Inc. 

1  ATTN:  Katherine  E.  Blythe 
275  Wyman  St. 

Waltham,  MA  02154 

Mr.  Albert  Stevens 

Bolt  Beranek  &  Newman,  Inc. 

1  10  Moulton  St. 

Cambridge,  MA  02238 

Mr.  Harvey  F.  Silverman 
Professor  of  Engineering 
Brown  University 
1  Providence,  RI  02912 

Di rector 

G/G-Training  Dynamics 
90  National  Drive 
1  Glastonbury,  CT  06033 

Analysis  &  Technology,  Inc. 

New  London  Office 
P.  0.  Box  769 

.  ATTN:  Frederick  M.  Ball 
New  London,  CT  06320 

Mystech  Associates,  Inc. 

P.  0.  Box  220 
Mystic,  CT  06355 
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Eclectech  Associates 
North  Stonington  Professional 
Center 

P.  0.  Box  178 

North  Stonington,  CT  06359 

Dr.  Thomas  J.  Hammell 
Eclectech  Associates 
P.  0.  Box  178 

North  Stonington,  CT  06333 

Mr.  W.  Bertsche 
Eclectech  Associates 
P.  0.  Box  178 

North  Stonington,  CT  06359 

Analysis  &  Technology,  Inc. 

ATTN:  Bonnie  E.  Jennings 
Technology  Park 
Route  2,  P.  0.  Box  220 
North  Stonington,  CT  06359 

Sonalysts,  Inc. 

215  Parkway  North 
Waterford,  CT  06385 

Dunlap  A  Associates,  Inc. 

One  Parkland  Drive 

ATTN:  J.  T.  Fucigna,  Executive  VP 

Darien,  CT  06820 

E.  P.  Washchilla 
Talley  Industries,  Research 
Center 

58  Progress  Drive 
P.  0.  Box  1456 
Stanford,  CT  06902 

Rockwell  International 
Autonetics  Marine  Systems  Div. 

New  England  Operations 
1028  Poquonnock  Rd. 

Groton,  CT  06340 

Lockheed  Electronics  Co.,  Inc. 
ATTN:  Support  Engineering  Dept. 
Mail  Station  1100 
1501  US  Highway  22,  C.  S.  #1 
Plainfield,  NJ  07061 

Systems  Engineering  Assocs.  Coro. 
P.  0.  Box  2030 

19  Cherry  Hill  Industrial  Park 
Perina  Blvd. 

Cherry  Hill,  NJ  08003 


1  Rowland  &  Co. 

P.  0.  Box  61 
Haddonfield,  NJ  08033 

Techplan  Corp. 

1000  Maplewood  Dr. 

3  Maple  Shade,  NJ  08052 

Geoffrey  M.  Heddon 
Computer  Sciences  Corp. 

P.  0.  Box  N 

1  Morrestown,  NJ  08057 

John  D.  Gould 

IBM  Research  Center 

Yorktown  Heights,  NY  10598 

Grumman  Aerospace  Corp. 
Plant  36,  C02-04 
ATTN:  Mr.  Sam  Campbell 
Bethpage,  LI,  NY  11714 

^  Martin  Plawsky 

Grumman  Aerospace  Corp. 
C21-05 

1  Bethpage,  NY  11714 

Glynn  R.  Ramsay 
Gould,  Inc. 

Simulation  Systems  Div. 

1  Melville,  NY  11747 

Martin  Morganlander 
Gould  SSD 
50  Marcus  Drive 
Melville,  NY  11747 

1  Howard  Jaslow 

Gould  Simulation  Systems 
50  Marcus  Dr. 

Melville,  NY  11747 

Harris  Corp. 

1  PRD  Electronics  Div. 

6801  Jerico  Turnoike 
Syosset,  NY  11791 

Eric/IR 

Syracuse  University 
,  School  of  Education 
1  Syracus,  NY  13210 
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Drs.  Cohen  and  Stark 
Link  Division 
The  Singer  Company 
Binghamton,  NY  13902 

Bob  Gamache 
Singer-Link  FSD 
Colesville  Rd. 

Binghamton,  NY  13902 

Alexandra  B.  Taylor 
Singer-Link  FSD 
Binghamton,  NY  13902 

SCIPAR,  Inc. 

ATTN:  Leslie  A .  Duffy 
P.  0.  Box  185 
Buffalo,  NY  14221 

Falcon  Research  &  Development  Co. 
ATTN:  Patricia  D.  Pfohl 
One  American  Drive 
Buffalo,  NY  14225 

Cal  span  Corporation 
4455  Genesee  St. 

Buffalo,  NY  14225 

George  R.  Purifoy,  Jr. 

Applied  Science  Assocs.,  Inc. 

Box  1 58 

Valencia,  PA  16059 
Dr.  M.  M.  Reddi 

Vice  President,  Engineering  Dept. 
Franklin  Research  Center 
20th  St.  4  "Benjamin  Franklin  Parkway 
Philadelphia,  PA  19103 

James  J.  Perriello 

Deputy  Div.  Mgr.,  Arlington  Dlv. 

Systems  Consultants,  Inc. 

2828  Pennsylvania  Ave. 

Washington,  DC  20007 

Charles  B.  Colton,  P.  E. 

Marketing  Oi rector 
Systems  Consultants,  Inc. 

2828  Pennsylvania  Ave. 

Washington,  DC  20007 


2  American  Institute  for  Research  1 

in  the  Behavioral  Sciences 
1055  Thomas  Jefferson  St.,  NW 
Washington,  DC  20007 

2  Robert  F.  Archer  1 

American  Institutes  for  Research 
in  the  Behavioral  Sciences 
1055  Thomas  Jefferson  St.,  NW 
Washinoton,  DC  20007 

1 

American  Psychological  Assoc.  1 

Psychological  Abstracts 
1200  17th  St.,  NW 
1  Washington,  DC  20036 

Robert  T.  Hennessy  1 

Committee  on  Human  Factors 
National  Research  Council 
1  2101  Constitution  Ave.,  NW 

Washington,  DC  20418 

TRACOR,  Inc.  1 

ATTN:  Training  Section  Head 
1  Two  Pine  Hill  Center 

P.  0.  Box  600 
Lexington  Park,  MD  20653 

1  GP  Technology  Corp.  1 

5615  Landover  Rd. 

Landover  Hills,  MD  20784 

B-K  Dynamics,  Inc.  1 

ATTN:  Barbara  Schardt 
1  MIS  Division 

15825  Shady  Grove  Rd. 

Rockville,  MD  20850 

E.  Scott  Baudhuin  2 

Singer-Link 

1  11800  Tech.  Rd. 

Silver  Spring,  MD  20904 

Richard  Kause  1 

Singer-Link 

Link  Simulation  Sys.  Div. 

2  11800  Tech.  Rd. 

Silver  Spring,  MD  20904 

ORI,  Inc.  1 

1400  Spring  St. 

Silver  Spring,  MD  20910 
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Vitro  Laboratories  Div. 

ATTN:  MO 

Automation  Industries,  Inc. 
14000  Georgia  Ave. 

Silver  Spring,  MD  20910 

Westinghouse  Electric  Corp. 
Defense  Group,  ILS  Div. 

ATTN:  Carole  W.  Forsythe 
MS  7740 

1111  Schilling  Road 
Hunt  Valley,  MD  21030 

Technical  Information  Center 
General  Physics  Corp. 

1000  Century  Plaza 
Columbia,  MD  21044 

Joyce  F.  Peacock,  Librarian 
AAI  Corp. 

Technical  Library 
P.  0.  Box  6767 
Baltimore,  MD  21204 

CADCOM 

ATTN:  Mr.  James  W.  Foust 
107  Ridgely  Ave. 

Annapolis,  MD  21401 

Ruth  Ehrensberger 
ARINC  Research  Corp. 

2551  Riva  Rd. 

Annapolis,  MD  21401 

American  Communications  Corp. 
Mail  Station-.B 
7617  Little  River  Turnpike 
Annadale,  VA  22003 

Synectlcs  Corp. 

ATTN:  Mr.  E.  M.  Jaehne 
10400  Eaton  Place 
Fairfax,  VA  22030 

Institute  of  Human  Performance 
Circle-Towers  Plaza  Level 
941 1 -R  Lee  Hiqhway 
Fairfax,  VA  22031 

Biotechnology,  Inc. 

ATTN:  Alma  Collie 
3027  Rosemary  Lane 
Falls  Church,  VA  22042 


1  Bert  Alton  1 

Sperry  Systems  Management 
12010  Sunrise  Valley  Drive 
Reston,  VA  22091 

Joseph  L.  Dickman  10 

Sperry  Systems  Management-SECOR 
1  12010  Sunrise  Valley  Dr. 

Reston,  VA  22091 

Library,  Tactical  Operations  1 

Analysis  Div. 

Science  Applications,  Inc. 

8400  Westpark  Drive 
1  McLean,  VA  22102 

Behavioral  Sciences  Department  1 

ATTN:  R.  J.  Ely 
The  BDM  Corporation 
1  7915  Jones  Branch  Drive 

McLean,  VA  22102 

PRO  Technical  Applications,  Inc.  1 
ATTN:  Karen  B.  Burklow 
7600  Old  Springhouse  Road 
1  McLean,  VA  22102 

Dr.  Edward  N.  Hobson  1 

The  BDM  Corporation 
7915  Jones  Branch  Drive 
1  McLean,  VA  22102 

General  Research  Corporation  1 

ATTN:  Mr.  Robert  Watt 
7655  Old  Springhouse  Road 
1  Westgate  Research  Park 
McLean,  VA  22102 

Advanced  Technology,  Inc.  1 

ATTN:  Debbie  .Burns 
1  Suite  500 

7923  Jones  Branch  Drive 
McLean,  VA  22102 

Alan  G.  Hundt,  PhD  1 

1  SAI  Comsys terns  Corporation 
7921  Jones  Branch  Drive 
McLean,  VA  22102 

Human  Sciences  Research,  Inc.  1 

1  Westgate  Research  Park 
7710  Old  Springhouse  Road 
McLean,  VA  22102 


NAVTRAEQUIPCEN  IH-341 


DISTRIBUTION 

Engineering  Research  Associates 
ATTN:  Erik  Thamm 
1517  Westbranch  Drive 
McLean,  VA  22102 

Jean  Hillman 

Advanced  Technology,  Inc. 

7923  Jones  Branch  Drive 
Suite  500 
McLean,  VA  22102 

George  H.  Datesman,  Jr. 

Program  Manager 

Applied  Science  Associates,  Inc. 

P.  0.  Box  975 
Middleburg,  VA  22117 

XMCO,  Inc. 

ATTN:  Director  of  Training 
Developments 

6501  Loisdale  Court,  Suite  606 
Springfield,  VA  22150 


Litton  Mel Ionics 

ATTN:  Ms.  Sue  Tepper,  Li  bn. 

P.  0.  Box  1286 
Springfield,  VA  22151 

Performance  Measurement  Assocs., 
Inc. 

131  Park  St.,  NE 
Vienna,  VA  22180 


Del  ex  Systems,  Inc. 

ATTN:  J.  Golden 

Suite  700,  8150  Leesburg  Pike 

Vienna,  VA  22180 

Raymond  G.  Fox 

Society  for  Appl ied- Learning 
Technology 
50  Culpeper  St. 

Warrenton,  VA  22186 

Analysis  &  Technology,  Inc. 

ATTN:  Bradley  Keyes 
Taylor  Building  (NC-3  -  Pentouse 
2531  Jefferson  Davis  Highway 
Arlington,  VA  22202 

Potomac  Research,  Inc. 

1600  North  Beauregard  St. 
Alexandria,  VA  22311 
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1  Rockwell  International  1 

ATTN:  D.  B.  Lafferman 
1745  Jefferson  Davis  Highway 
Suite  1107 

Arlington,  VA  22202 

1 

Mantech  International  1 

Century  Building 
2341  Jefferson  Davis  Highway 
Arlington,  VA  22202 

1  W.  Kessler  1 

Data-Design  Laboratories 
Suite  307 

1755  South  Jefferson  Davis  Highway 
Arlington,  VA  22202 

^  SRI  International  1 

1611  North  Kent  St. 

Arlington,  VA  22209 


Don  R.  Lyon  1 

1  Editor-in-Chief 
Mews  &  Motes 

Personnel  A  Training  Research 
GrouD 

Office  of  Naval  Research 
1  Arlington,  VA  22217 

Control  Data  Corporation  1 

ATTN:  Mr.  W.  W.  Doescher 
1800  North  Beauregard  St. 
Alexandria,  VA  22311 

Dr.  Edgar  L.  Shriver,  President  1 
Kinton  Incorporated 
Suite  205 

1500  North  Beauregard  St. 
Alexandria,  VA  22311 


Director 

Human  Resources  Research  1 

Organization 

1  300  N.  Washington  St. 

Alexandria,  VA  22314 

Allen  Corp.  of  America  1 

ATTN:  W.  K.  Rugemer,  Jr. 

401  Wythe  St. 

1  Alexandria,  VA  2231  ^ 
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Essex  Corporation 

ATTN:  Dr.  Thomas  B.  Malone 

333  N.  Fairfax  St. 

Alexandria,  VA  22314 

Dr.  Saul  Lavisky,  Executive  Officer 
HumRRO 

300  N.  Washington  St. 

Alexandria,  VA  22314 

Q.  E.  D.  Systems,  Inc. 

ATTN:  Special  Projects 
4646  Witchduck  Rd. 

Virginia  Beach,  VA  23455 

Comptek  Research,  Inc. 

#5  Kroger  Executive  Center,  Suite  127 
Norfolk,  VA  23502 

Janet  L.  Winner 
4208  Colonial  Ave. 

Norfolk,  VA  23508 

Dr.  Robert  C.  Williges 
Industrial  Eng.  &  Operations  Rsch. 
Virginia  Polytechnic  Inst.  &  State 
University 

Blacksburg,  VA  24061 

R.  L.  Parker  1 

Lockheed  Georgia  Co. 

Customer  Tng.  Dept. 

86  S.  Cobb  Dr. 

Marietta,  GA  30063 

Dr.  Ross  Gagliano  1 

Head,  Operations  Research  Branch 
Georgia  Tech.  Eng.  Experiment  Station 
326  Baker  Bldg. 

Atlanta,  GA  30332 

David  M.  Eakin  1 

Bell  Aerospace  Textron 
Naval  Coastal  Systems  Center 
Bldg.  319 

Panama  City,  FL  32407 

B.  L.  Nissley  1 

Bell  Aerospace  Textron 
Naval  Coastal  Systems  Center 
Panama  City,  FL  32407 


Georgia  Institute  of  Technology  1 
Computer  Science  &  Tech.  Lab. 
Engineering  Experiment  Station 
Atlanta,  GA  30332 

B.  E.  Mulligan  1 

University  of  Georgia 
Dept  of  Psychology 
Athens,  GA  30602 

Paul  W.  Caro  3 

Seville  Research  Corp. 

400  Plaza  Bldg. 

Pensacola,  FL  32505 

Seville  Research  Corporation  1 

Pace  Blvd.  at  Fairfield 
Pensacola,  FL  32505 

John  T.  McShera,  Jr.  1 

2112  Lewis  Turner  Blvd. 

Ft.  Walton  Beach,  FL  32548 

G.  H.  Balz  1 

Balz  Enterprises,  Inc. 

2524  Chinook  Trail 
Maitland,  FL  32751 

Hr.  A.  F.  Plogstedt,  M/S  100  1 

McDonnell  Douglas  Astronautics  Co. 

P.  0.  Box  600 
Titusville,  FL  32780 

R.  Fagan  1 

250  W.  Lake  Sue  Ave. 

Winter  Park,  FL  32798 

John  P.  Quinn  1 

NSI 

1315  S.  Semoran  Blvd. 

Winter  Park,  FL  32792 

Grace  P.  Waldrop  1 

McDonald  &  Associates,  Inc. 

988  Woodcock  Rd. 

Suite  136 

Orlando,  FL  32803 

Bruce  McDonald 
McDonald  &  Associates 
988  Woodcock  Rd. 

Orlando.  FL  32803 
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Appl i-Mation,  Inc. 

31291  Maguire  Blvd. 

Suite  244 

Orlando,  FL  32803 

Michael  Skurzewski 
Veda,  Inc. 

3203  Lawton  Rd. 

Suite  233 

Orlando,  FL  32303 

Jim  Hazzard 
Veda,  Inc. 

3203  Lawton  Rd. 

Suite  233 

Orlando,  FL  32803 
John  Keegan 

Allen  Corporation  of  America 
3751  Maguire  Bldg. 

Suite  270 

Orlando,  FL  32803 
Bob  Baker 

Allen  Corporation  of  America 
999  Woodcock  Rd. 

Suite  308 

Orlando,  FL  32803 

Science  Applications,  Inc. 
3655  Maguire  Blvd. 

Suite  150 

Orlando,  FL  32803 

Diana  Z.  Bryan 
Eagle  Technology 
3165  McCrory  pi . ,  STE.  235 
Orlando,  FL  32803 


Deborah  Bush 
E-Tech,  Inc. 

3165  McCrory  PI. 
Orlando,  FL  32803 

William  Hinton 
Spectrum  of  America 
1040  Woodcock  Rd. 
Suite  214 

Orlando,  FL  32803 

Moses  Aronson 
Aronson  Industries 
3705  Wilder  Lane 
Orlando,  FL  32804 


1  Wal ter  M.  Konanski  2 

Spectrum  of  America 
1040  Woodcock  Rd. 

Suite  214 

Orlando,  FL  32803 

1 

Northrop  Services,  Inc.  1 

ATTN:  G.  F.  Stackpole 
3203  Lawton  Road 
Suite  201 

Orlando,  FL  32803 


Harold  Rosenblum  1 

1310  Webster  St. 

Orlando,  FL  32804 

Martin  A.  Traub  1 

2  Appli-Mation,  Inc. 

6955  Hanoi ng  Moss  Road 
Orlando,  FL  32807 

Dave  Daly  1 

Rowland  &  Co. 

1  6961  Hanging  Moss  Rd. 

Orlando,  FL  32807 

Diana  Barry  1 

Rowland  &  Co. 

6961  Hanning  Moss  Rd. 

1  Orlando,  FL  32807 

Patricia  Kilgore  1 

Rowland  %  Co. 

6961  Hangino  Moss  Rd. 

1  Orlando,  FL'  32807 


Dr.  Angelo  P.  Verdi  1 

Rowland  &  Co. 

6961  Hanging  Moss  Rd. 

1  Orlando,  FL  32807 

R.  W.  Schultheis,  Jr.  1 

Computer  Systems  Division 
Perkin-Elmer 

1  7200  Lake  Ellenor  Dr. 

Orlando,  FL  32809 

Jeffery  L.  Maxey  1 

3165  McCrory  PL 
Orlando,  FL  32813 

2 

Ergonomics  Associates,  Inc.  1 

P.  0.  Box  20987 
Orlando,  FL  32814 
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DBA  Systems,  Inc. 

ATTN:  Advanced  Programs 
P.  0.  Drawer  550 
Melbourne,  FL  32901 

T.  F.  NcCrea 
Chrysler  Defense 
2423  Auburn  Dr. 

Cocoa,  FL  32922 

DEACO  International ,  Inc. 
Intercontinental  Bank  Bldg. 

3899  N.  W.  7th  St.,  Suite  211 
Miami,  FL  33126 

Thomas  P.  Ross,  Jr. 

Science  Applications,  Inc. 

2109  W.  Clinton  Ave. 

Suite  800 

Huntsville,  AL  35805 
Rudolf  K.  Walter 

International  Aerospace  Technologies 
Inc. 

Executive  Plaza,  Suite  103 
4717  University  Drive 
Huntsville,  AL  35805 

Dr.  Richard  S.  Jensen 
Aviation  Psychology  Lab. 

Dept,  of  Aviation 

The  Ohio  State  University 

Box  3022 

Columbus,  OH  43210 

Joseph  R.  Gleydura 
Goodyear  Aerospace  Corp. 

1210  Massillon  Rd. 

Akron,  OH  44315 

George  E.  Paler 
Goodyear  Aerospace  Corp. 

1210  Massillon  Rd. 

Akron,  OH  44315 

Donald  K.  Oldham 
Goodyear  Aerospace 
1210  Massillon  Rd. 

Akron,  OH  44315 

Simulation  Technology,  Inc. 

ATTN:  R.  B.  Penwell 
4124  Linden  Ave.,  Suite  200 
Dayton,  OH  45432 


1  John  M.  Howard  1 

Staff  Scientist 
Human  Factors  Oiv. 

SSS  Consulting 
2600  Far  Hills 

2  Dayton,  OH  45419 

Carmine  Vaccarino  1 

SRL,  Inc. 

2800  Indian  Ripple  Rd. 

'  Dayton,  OH  45440 

Systems  Research  Lab.  Inc.  1 

ATTN:  George  A.  Whiteside 
2800  Indian  Ripple  Rd. 

1  Dayton,  OH  45^40 

James  R.  Bridges  3 

Program  Mgr.  Tng.  Equipment 
General  Dynamics 
Land  Systems  Div. 

1  P.  0.  Box  1901 

Warren,  Michigan  48090 

Bjorksten  Research  Laboratories,! 
Inc. 

p.  0.  Box  9444 

,  ATTN:  Mr.  Michael  J.  Maloney 
Madison,  WS  53715 

John  F.  Brock  2 

Systems  &  Research  Center 
Honeywell,  Inc. 

2600  Ridgeway  Parkway 

3  Minneapolis,  MN  55413 

Tony  Modrick  1 

Honeywell  Systems  &  Research  Center 
P.  0.  Box  312  . 

■,  Minneapolis,  MN  55440 

J.  i.  Runquist  1 

Market  Analyst 
Sperry  Uni  vac 
1  Univac  Park 
P.  0.  Box  3525 
St.  Paul,  MN  55165 

McDonnell  Douglas  Astronautics  Co.  i 
1  ATTN:  E.  J.  Reilly 

Dept.  E080,  Bldg  106,  Level  *. 

P.  0.  Box  516 
St.  Louis,  MO  63166 
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John  W.  Robbins  1  Pete  Chism  1 

McDonnell  Douglas  Astronautics  Co.  Bell  Aerospace  Textron 
Dept.  E502,  Bldg  81,  Level  2,  Post  El 0  6800  Plaza  Dr. 

Box  516  New  Orleans,  LA  70127 

St.  Louis,  MO  63166 

John  Priest  1 

J.  Germeroth  1  1651  Rockwood  Trail 

McDonnell  Douglas  Electronics  Co.  Fayetteville,  AR  72701 

2600  N.  Third  St. 

St.  Charles,  MO  63301  Lynn  R.  Whisman  1 

Burtek,  Inc. 

Ross  E.  Ailslieger  1  P.  0.  Box  1677 

Crew  Systems  Technology  -  M.S.  K36-40  Tulsa,  OK  74101 


Boeing  Military  Airplane  Co. 

3801  S.  Oliver  St. 

Wichita,  KS  67210 

Robert  B.  Grow 
Simufl ite  Tng.  Inti . 

7610  Stemmons 
Suite  300 
Dallas,  TX  75247 

V.  S.  Perkins,  Jr. 

Vaught  Corporation 
P.  0.  Box  225907 
M.  S.  31-02 
Dallas,  TX  75265 

Dr.  Ronald  R.  Calkins 
Vaught  Corporation 
Bldg.  31 

P.  0.  Box  225907 
Dallas,  TX  75265 

General  Dynamics/ Fort  Worth 
ATTN:  J.  A.  Davis 
P.  0.  Box  746,  MZ  2865 
Fort  Worth,  TX  76101 

Terry  Nagle 

Bell  Aerospace  Textron 

6800  Plaza  Dr. 

New  Orleans,  LA  70127 
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